Systems, methods, and computer readable media for using descriptors to identify when a subject is likely to have a dysmorphic feature

ABSTRACT

Systems, methods, and computer-readable media are disclosed for identifying when a subject is likely to be affected by a medical condition. For example, at least one processor may be configured to receive information reflective of an external soft tissue image of the subject. The processor may also be configured to perform an evaluation of the external soft tissue image information and to generate evaluation result information based, at least in part, on the evaluation. The processor may also be configured to predict a likelihood that the subject is affected by the medical condition based, at least in part, on the evaluation result information.

RELATED APPLICATIONS

This application is a continuation-in-part of International ApplicationNo. PCT/IB2014/001235, filed Mar. 12, 2014, which claims priority toU.S. Provisional Application No. 61/778,450, filed Mar. 13, 2013, bothof which are incorporated herein by reference in their entirety.

TECHNICAL FIELD

The present disclosure relates to the field of image analysis. Forexample, systems, methods, and computer-readable media are disclosed foridentifying when a subject is likely to be affected by a medicalcondition using image analysis.

BACKGROUND

There are thousands of known rare diseases that collectively affect morethan 8% of the world's population. Rare diseases are often chronic,progressive, degenerative, and life threatening. Children affected byrare diseases often suffer from many associated medical complicationsand need critical and timely medical intervention.

Many rare diseases are genetic in origin, inborn, and exhibitsymptomatic malformations. Symptomatic malformations are often the firstsign of a rare disease. A dysmorphic evaluation performed by a qualifiedspecialist often plays a key factor in recognizing a disease. But due tothe rarity of many diseases, the scarcity of dysmorphology experts, andthe complexity of a clinical diagnosis, is it often not possible toprovide proper and comprehensive dysmorphology training to a largenumber of physicians. The diagnosis of rare diseases is often verydifficult, particularly for physicians that lack the relevant awareness,knowledge, and experience. Most children that do reach a diagnosis aretypically diagnosed later in life when physical symptoms, developmentaldelay, intellectual disability, and other medical complications areobserved by their families or treating physician. This can result in anunmanaged and untreated disease that can cause a child's condition todeteriorate.

Early identification of diseases is often critical. Accordingly, systemsand methods are needed that can efficiently and noninvasively determinewhether a person is likely to be affected by a medical condition.

SUMMARY

Embodiments consistent with the present disclosure provide systems,methods, and computer-readable media for identifying when a subject islikely to be affected by a medical condition using image analysis.

In one disclosed embodiment, a system for identifying when a subject islikely to be affected by a medical condition is disclosed. The systemincludes at least one processor that is configured to receiveinformation reflective of an external soft tissue image of the subject,perform a first evaluation of the external soft tissue image informationusing at least one of an anchored cells analysis, a shifting patchesanalysis and a relative measurements analysis, generate first evaluationresult information based, at least in part, on the first evaluation,perform a second evaluation of the external soft tissue imageinformation using at least one of the anchored cells analysis, theshifting patches analysis, and the relative measurements analysis,generate second evaluation result information based, at least in part,on the second evaluation, and predict a likelihood that the subject isaffected by the medical condition based, at least in part, on the firstevaluation result information and the second evaluation resultinformation.

In another disclosed embodiment, a system for identifying when a subjectis likely to be affected by a medical condition is disclosed. The systemincludes at least one processor that is configured to receiveinformation reflective of an external soft tissue image of the subject,divide the external soft tissue image information into a plurality ofregions, generate an analysis of each of the plurality of regions,aggregate the analyses of the plurality of regions, and determine alikelihood that the subject is affected by the medical condition basedon the aggregated analyses.

In another disclosed embodiment, a system for identifying when a subjectis likely to be affected by a medical condition is disclosed. The systemincludes at least one processor that is configured to receiveinformation reflective of an external soft tissue image of the subject,use image information analysis to compare the external soft tissue imageinformation with a plurality of external soft tissue images of othersubjects in a database, determine, based on the image informationanalysis, dysmorphic features included in the external soft tissue imageinformation, access descriptors associated with the dysmorphic features,and output at least some of the descriptors.

In another disclosed embodiment, a system for identifying when a subjectis likely to be affected by a medical condition is disclosed. The systemincludes at least one processor that is configured to receiveinformation reflective of an external soft tissue image of the subject,analyze the external soft tissue image information, identify one or moreexternal soft tissue attributes in the external soft tissue imageinformation based, at least in part, on the analysis, access at leastone database of external soft tissue attributes associated with aplurality of medical conditions, compare the one or more identifiedexternal soft tissue attributes with the external soft tissue attributesof the at least one database, and output information about at least onemedical condition likely possessed by the subject based on thecomparison.

In another disclosed embodiment, a system for identifying when a subjectis likely to be affected by a medical condition is disclosed. The systemincludes at least one processor that is configured to receive firstinformation reflective of a first external soft tissue image of thesubject recorded at a first time, analyze the first image information,receive second information reflective of a second external soft tissueimage of the subject recorded at a second time, analyze the second imageinformation, compare the analysis of the first image information withthe analysis of the second image information, and predict a likelihoodthat the subject is affected by the medical condition based, at least inpart, on the comparison.

Additional aspects related to the disclosed embodiments will be setforth in part in the description which follows, and in part will beunderstood from the description, or may be learned by practice of thedisclosed embodiments.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this disclosure, illustrate various disclosed embodiments. Inthe drawings:

FIG. 1 illustrates an example system for identifying when a subject islikely to be affected by a medical condition that may be used forimplementing the disclosed embodiments.

FIG. 2 illustrates example operations that a processor of a medicalcondition analysis system may be configured to perform to predict alikelihood that a subject is affected by a medical condition using twoevaluations, in accordance with some of the disclosed embodiments.

FIG. 3 illustrates example operations that a processor of a medicalcondition analysis system may be configured to perform to predict alikelihood that a subject is affected by a medical condition using animage division, in accordance with some of the disclosed embodiments.

FIG. 4 illustrates example operations that a processor of a medicalcondition analysis system may be configured to perform to predict alikelihood that a subject is affected by a medical condition usinginformation of at least one relative of the subject, in accordance withsome of the disclosed embodiments.

FIG. 5 illustrates example operations that a processor of a medicalcondition analysis system may be configured to perform to output atleast some descriptors associated with dysmorphic features, inaccordance with some of the disclosed embodiments.

FIG. 6 illustrates example operations that a processor of a medicalcondition analysis system may be configured to perform to predict alikelihood that a subject is affected by a medical condition using atleast one hundred defined locations on an image, in accordance with someof the disclosed embodiments.

FIG. 7 illustrates example operations that a processor of a medicalcondition analysis system may be configured to perform to superimposeindicates of at least one dysmorphology on an image, in accordance withsome of the disclosed embodiments.

FIG. 8 illustrates example operations that a processor of a medicalcondition analysis system may be configured to perform to identifyinformation about dysmorphic features in a selected region, inaccordance with some of the disclosed embodiments.

FIG. 9 illustrates example operations that a processor of a medicalcondition analysis system may be configured to perform to outputinformation about at least one medical condition likely possessed by asubject, in accordance with some of the disclosed embodiments.

FIG. 10 illustrates example operations that a processor of a medicalcondition analysis system may be configured to perform to predict alikelihood that a subject is affected by a medical condition based onanalyses at two different times, in accordance with some of thedisclosed embodiments.

FIG. 11 illustrates example operations that a processor of a medicalcondition analysis system may be configured to perform to determine apreviously unrecognized medical condition likely possessed by twosubjects, in accordance with some of the disclosed embodiments.

FIG. 12 illustrates example operations that a processor of a medicalcondition analysis system may be configured to perform to mediatecommunications between a health service provider and a healthcareprofessional, in accordance with some of the disclosed embodiments.

FIG. 13 illustrates example operations that a processor of a medicalcondition analysis system may be configured to perform to alert ahealthcare provider when an image of a subject meets a threshold ofbeing likely to be affected by a medical condition, in accordance withsome of the disclosed embodiments.

FIG. 14 illustrates example operations that a processor of a medicalcondition analysis system may be configured to perform to predictwhether a subject has a medical condition, in accordance with some ofthe disclosed embodiments.

FIG. 15 illustrates example operations that a processor of a medicalcondition analysis system may be configured to perform to generate alist of tests to be performed, in accordance with some of the disclosedembodiments.

FIG. 16 illustrates example operations that a processor of a medicalcondition analysis system may be configured to perform to predictwhether a dysmorphic feature is indicative of a medical condition basedon a severity score, in accordance with some of the disclosedembodiments.

FIG. 17 illustrates example operations that a processor of a medicalcondition analysis system may be configured to perform to predictwhether a subject is likely to be affected by a medical condition bydiscounting at least one dysmorphic feature, in accordance with some ofthe disclosed embodiments.

FIG. 18 illustrates exemplary depictions of an image processing pipelinein accordance with some of the disclosed embodiments.

FIGS. 19-22 illustrate exemplary depictions of image segmentation inaccordance with some of the disclosed embodiments.

FIGS. 23A-23C illustrate exemplary depictions of an anchored cellanalysis in accordance with some of the disclosed embodiments.

FIG. 24 illustrates exemplary depictions of a shifting patch analysis inaccordance with some of the disclosed embodiments.

FIG. 25 illustrates exemplary depictions of a relative measurementsanalysis in accordance with some of the disclosed embodiments.

FIG. 26 illustrates exemplary depictions of a plurality of analyses inaccordance with some of the disclosed embodiments.

FIG. 27 illustrates exemplary depictions of an ear analysis inaccordance with some of the disclosed embodiments.

FIG. 28 illustrates exemplary depictions of an undiagnosed patientanalysis in accordance with some of the disclosed embodiments.

FIG. 29 illustrates exemplary depictions of a vector comparison analysisin accordance with some of the disclosed embodiments.

DETAILED DESCRIPTION

Reference will now be made in detail to the example embodiments, whichare illustrated in the accompanying drawings. Wherever possible, thesame reference numbers will be used throughout the drawings to refer tothe same or like parts.

FIG. 1 is a diagram illustrating an example system 100 for identifyingwhen a subject is likely to be affected by a medical condition,consistent with the disclosed embodiments. A subject may include, amongother things, any person or type of person, such as a male or femaleperson and a child or adult. A child may include, for example, aneonate, an infant, a toddler, a preschooler, a school age child, or anadolescent. For example, a male or female person from birth to 1 monthold may be referred to as a neonate, from 1 month to 1 year old may bereferred to as an infant, from 1 year to 3 years old may be referred toas a toddler, from 3 years to 6 years old may be referred to as apreschooler, from 6 years to 12 years old may be referred to as a schoolage child, and from 12 years to 18 years old may be referred to as anadolescent. An adult may include, for example, a male or female personfrom 18 years old and onwards. These age ranges, however, are exemplaryonly. For example, a 19 year old person may be referred to as anadolescent in certain contexts.

A medical condition may include, among other things, any medicaldisease. A subject possessing a medical condition may include, forexample, at least one of possessing a genetic syndrome and being acarrier of a genetic syndrome. A medical condition may also include,among other things, any association of clinically recognizable features,signs, symptoms, phenomena, or other characteristics that often occurtogether, such that the presence of one feature, sign, symptom,phenomena, or other characteristic may imply, indicate, or alert to thepossible presence of the others. A medical condition may also includeone or more abnormal findings in physical growth and development overtime (e.g., growth deficiencies and craniofacial deformations thatdevelop over time). For example, a medical condition may be one or moreof the medical conditions disclosed in “Gorlin's syndromes of the headand neck,” 2010, Oxford University Press, to R.C.M. Hennekam et al, “TheBedside Dysmorphologist,” 2008, Oxford University Press, to WilliamReardon, and “Smith's Recognizable Patterns of Human Malformation,”2005, WB Saunders, to Kenneth Lyons Jones, all of which are incorporatedherein by reference in their entirety.

In some embodiments, a medical condition includes one or more conditionsthat may cause a person to exhibit one or more dysmorphic features. Adysmorphic feature may include, for example, any feature that affectsthe appearance of a subject. A dysmorphic feature may, for example,reflect an external soft tissue dysmorphology. For example, a medicalcondition may cause a child's skull to form in an irregular manner,which may cause the child's facial appearance to also be irregular in amanner that may be described by one or more dysmorphic features. Forexample, a dysmorphic feature may be one or more of the dysmorphicfeatures disclosed in “Elements of morphology: Introduction,” 2009, Am JMed Genet Part A 149A:2-5, to Allanson et al., “Elements morphology:Standard of terminology for the head and face,” 2009, Am J Med GenetPart A 149A:6-28, to Allanson et al., “Elements of morphology: Standardterminology for the lips, mouth, and oral region,” 2009, Am J Med GenetPart A 149A:77-92, to Carey et al., “Elements of morphology: StandardTerminology for the periorbital region,” 2009, Am J Med Genet Part A149A:29-39, to Hall et al., “Elements of morphology: Standardterminology for the Nose and philtrum,” 2009, Am J Med Genet Part A149A:61-76, to Hennekam et al., “Elements of morphology: Standardterminology for the ear,” 2009, Am J Med Genet Part A 149A:40-60, toHunter et al., and “Elements of morphology: Standard terminology for thehands and feet,” 2009, Am J Med Genet Part A 149A:93-127, to Bieseckeret al., all of which are incorporated herein by reference in theirentirety.

Further, when the medical condition is a genetic disorder, predictingthe likelihood that the subject is affected by the genetic disorder, asused herein, may include, among other things, the ability to screen fora genetic disorder, reach a definitive or differential diagnosis, ruleout a diagnosis, monitor the progression of a genetic disorder, describethe natural history of a genetic disorder, define and evaluate thephenotype associated with a genetic disorder, correlate phenotypicattributes to genetic variants, discover biomarkers for a geneticdisorder, facilitate development of new therapies or repurpose existingtherapies and test their efficacy, recruit and match candidates forclinical trials, facilitate development of new diagnostic tools andmethods, and facilitate development of new or improved bioinformaticssolutions. Genetic disorders may include, among other things, a medicalcondition caused by one or more abnormalities or variants in the genome,including a condition that is present from birth (congenital). Geneticdisorders may or may not be heritable. In non-heritable geneticdisorders, defects may be caused by new mutations or changes to the DNA.

System 100 may include, among other things, at least one processor 110,at least one memory device 120, at least one input device 130, at leastone camera 140, and at least one output device 150. Processor 110 mayinclude any electrical circuit (e.g., processing circuitry) that may beconfigured to perform an operation on at least one input variable,including, for example one or more integrated circuits, microchips,microcontrollers, and microprocessors, which may be all or part of acentral processing unit (CPU), a digital signal processor (DSP), a fieldprogrammable gate array (FPGA), a graphical processing unit (GPU), orany other circuit known to those skilled in the art that may be suitablefor executing instructions or performing logic operations. As such, theprocessor 110 may be referred to as processing circuitry in someembodiments.

Multiple functions may be accomplished using a single processor ormultiple related and/or unrelated functions may be divided amongmultiple processors. Processor 110 may be configured to access memorydevice 120, which may include, for example, persistent memory, ROM,EEPROM, EAROM, flash memory devices, magnetic disks, magneto opticaldisks, CD-ROM, DVD-ROM, Blu-ray, and the like. Memory device 120 maycontain instructions (i.e., software or firmware) or other data, whichthe processor 110 may be configured to execute. As such, someembodiments of the methods presented herein may be stored on one or morememory devices 120 as computer-executable instructions that may beexecuted by the processor 110 or any other electrical computer orcircuitry. Processor 110 may receive instructions and data stored memorydevice 120. Thus, in some embodiments, processor 110 may execute thesoftware or firmware to perform functions by operating on input data andgenerating output. However, processor 110 may also receive or accessdata stored remotely over a network (not depicted in FIG. 1). Forexample, device 100 may include a communication device (not depicted inFIG. 1) that enables processor 110 to receive or access data storedremotely on a server or user device over a network. Moreover, processor110 may also be, for example, dedicated hardware or an applicationspecific integrated circuit (ASIC) that performs processes by operatingon input data and generating output. Processor 110 may be anycombination of dedicated hardware, one or more ASICs, one or moregeneral purpose processors, one or more DSPs, one or more GPUs, or oneor more other processors capable of processing digital information. Forexample, in some embodiments, processor 110 may comprise multipleprocessors that may provide parallel processing capabilities.

FIG. 2 illustrates an exemplary process 200 that at least one processormay be configured to perform. For example, as discussed above, processor110 may be configured to perform process 200 by executing software orfirmware stored in memory device 120, or may be configured to performprocess 200 using dedicated hardware or one or more ASICs.

Processor 110 may be configured to receive information reflective of anexternal soft tissue image of the subject (step 210). For example,processor 110 may receive information, such as pixel values, reflectiveof an external soft tissue image of a subject captured by camera 140.Camera 140 may include, among other things, one or more image sensors,such as a CCD image sensor, a CMOS image sensor, a camera, a lightsensor, an IR sensor, an ultrasonic sensor, a proximity sensor, ashortwave infrared (SWIR) image sensor, a reflectivity sensor, or anyother image sensor that is capable of capturing an external soft tissueimage. An image sensor may be configured to capture any quantity ofimage data, such as single pixel data, one-dimensional line data,two-dimensional data, or three-dimensional data. Camera 140 may be afixed camera, mobile camera, or any other image capturing device orequipment, which may, for example, be further incorporated into acomputer, a tablet, a phone, glasses, or any other device.

An external soft tissue image may include, among other things, an image,such as a digital image comprised of pixels, of a subject or any portionof a subject. In some embodiments, the external soft tissue image mayinclude an image of at least one of a face of the subject, a cranium ofthe subject, a hand of the subject, and a foot of the subject. In suchembodiments, the external soft tissue image may be referred to as a typeof external image of the subject, for example, an external cranio-facialsoft tissue image, which may be an external soft tissue image of thecranio-facial portion of the subject. However, the external soft tissueimage may also include other portions of the subject, such as, such as ahairline, forehead, ocular region, eyebrow, nose, eye, mid-face region,philtrum region, mouth, ear, mandibular region, chin, cheek, neck,chest, mid-body, back, torso, hips, genitalia, limbs, joints, hands, andfingers. In some embodiments, the external soft tissue image is acranio-facial image that includes at least one of a frontal view, alateral view, an angled view, a top view, and a back view. As usedherein, a cranio-facial image is an image that includes at least aportion of a cranium or face of the subject. A frontal view may includean image of the front of the face of the subject. A lateral view mayinclude an image taken at or approximately at a 20-90 degree angle (tothe left and/or right side of the face) from the vertical midline of thehead of the subject. For example, in one embodiment, the lateral viewmay include an image taken at or approximately at a 45 degree angle (tothe left and/or right side of the face) from the vertical midline of thehead of the subject. A top view may include an image of the top of thehead of the subject. A back view may include an image of the back of thehead of the subject. As described in more detail below, in someembodiments the external soft tissue image is associated with adysmorphology.

The information reflective of an external soft tissue image received byprocessor 110 may include the external soft tissue image itself or anydata derived from the external soft tissue image (e.g., a separateprocessor at camera 140 may derive data from the external soft tissueimage and transmit the derived data to processor 110). For example, ifthe external soft tissue image is an analog image (although the externalsoft tissue image may be captured as a digital image), informationreflective of an external soft tissue image may include a digitallyconverted version of the external soft tissue image. The informationreflective of an external soft tissue image may be, for example, avector image or a raster image. The information reflective of anexternal soft tissue image may also be non-image data, such as a set ofparameters derived from the external soft tissue image, which mayinclude, for example, one or more intensities of the image, one or morelocations of edges in the image, and one or more textures in the image.

In some embodiments, processor 110 may preprocess the external softtissue image information as it receives the information and/or after itreceives the information. FIG. 18 depicts one example of a preprocessingroutine that may be performed by processor 110. As depicted in FIG. 18,as part of a preprocessing routine, processor 110 may be configured todetect a face region of the external soft tissue image information,detect a number of points in the face region, and align the face region.One example of a face detection routine is shown in FIGS. 19-22. Forexample, as graphically depicted in FIG. 20, processor 110 may beconfigured to detect a face region by first placing a plurality ofpatches (i.e., sub-regions of the image information), which optionallymay overlap one another, over the image information. For each patch, adescriptor vector may be computed. A descriptor vector may include, forexample, data derived from at least one of a scale-invariant featuretransform (SIFT), a histogram of oriented gradients (HOG), aself-similarity descriptor, a histogram of Local Binary Patterns, adescriptor based on the activation of at least one layer in a neuralnetwork, and any other determinable feature known in the image analysisand computer vision fields.

During a training phase for the face detection routine, one or moreregions of a set of training images may be manually outlined. Forexample, processor 110 may determine an outline of a head and one ormore regions within a face or side of a face (e.g., an outline of eyes,nose, mouth, ears, etc.). Processor 110 may determine a center of massof the outline of the head (e.g., a point where the weighted relativeposition of the points on the outline sums to zero). Processor 110 mayfurther determine a descriptor vector for each patch of each trainingimage and store the descriptor vectors in a database. The database maybe stored by memory device 120, or may be stored on, for example, aremote server that is accessible to processor 110 over a network.Moreover, information regarding the location of the center of mass ofthe head shape relative to the center of each patch associated with adescriptor vector may also be stored in the database. FIG. 19 depicts anexample of an outlined head shape with information regarding thelocation of the center of mass of the head shape relative to the centerof a plurality of patches.

Further regarding the face detection routine, processor 110 may beconfigured to compare the descriptor vector of each patch in the imageinformation of the subject to descriptor vectors associated with patchesof training images to determine a set of the most similar descriptorvectors (e.g., the 25 most similar descriptor vectors). To perform thecomparison, processor 110 may be configured to retrieve the descriptorvector of each patch in the image information from memory device 120and/or from a remote source over a network (e.g., a server or userdevice), and may be configured to retrieve the descriptor vectorsassociated with patches of training images from memory device 120 and/orfrom a remote source over a network (e.g., a server or user device).Processor 110 may be configured to calculate one or more of a Euclideandistance, a Chebyshev distance, a chi-square distance, and a Mahalanobisdistance between the descriptor vector of each patch in the imageinformation of the subject and the descriptor vectors associated withpatches of training images. The set of the most similar descriptorvectors may include, for example, those descriptor vectors that areassociated with a shortest Euclidean distance, Chebyshev distance,chi-square distance, or Mahalanobis distance. The patches associatedwith the set of the most similar descriptor vectors may be retrievedfrom the database. Each patch retrieved from the database may provideone vote for a location of the center of mass of the image informationof the subject. A vote may be determined by adding a relative locationof the center of mass in the training image associated with a givenpatch to the location of the patch in the image information used toretrieve the patch from the database. A vote, as used in this context,refers to one estimate for a location of the center of mass of the imageinformation of the subject. All votes from all patches of the imageinformation may be integrated and the location with the most support maybe selected as the center of mass of the head. FIG. 20 graphicallydepicts an example of the described voting operations.

In some embodiments, for each patch in the image information of thesubject, patches in the set of retrieved patches that point to theselected center of mass location and are within a threshold distancefrom the selected center of mass may be discarded. The rest are assigneda score that is proportional to two elements: the similarity between theretrieving patch descriptor vector and the retrieved patch descriptorvector, and a distance between the selected center of mass and thecenter of mass implied by the retrieved patch. For each retrieving patch(i.e., a patch in the image information of the subject), the scores ofthe retrieved patches may be accumulated. A threshold filter may beapplied to the accumulated score of each of the retrieving patches toobtain an initial rough estimate of a foreground region of the headshape in the image information. Processor 110 may be configured to applyone or more morphological operations on the initial rough estimate toproduce a closed shape. As graphically depicted, for example, in FIG.21, the contour of the closed shape may serve as a first segmentationhypothesis. In some embodiments, processor 110 may also be configured toapply a mean shift segmentation algorithm and/or a GrabCut segmentationalgorithm to the image information using the first segmentationhypothesis as a starting position for the computation.

In some embodiments, processor 110 may use the contour described aboveas the detected head or face region. However, in some embodiments, thedetermination is further refined. For example, in some embodiments acontour of each training image may be represented by a vector containingone or more for the following: (i) the (x, y) coordinates of a number ofpoints (e.g., 50) sampled along the contour at equal distance, whereinthe first point is taken, for example, to be the top most point alongthe contour; (ii) the location of the first point as well as thedifferences in (x, y) coordinates between every point and the next point(e.g., a number of pairs of (dx, dy) summing up to zero); (iii) thedistance of each such contour point from the center of the mass of thecontour's captured area and the angle of the ray from the center of massto a contour point; and (iv) the distances from the center of mass tothe points on the contour. To refine the estimated contour of the imageinformation, processor 110 may employ a Principal Components Analysis tocompute the Principal Components of the training vectors. The estimatedcontour may be refined by projecting it onto the space of the PrincipalComponents.

Processor 110 may identify regions in which one or more of thepreviously determined contours are consistent. For example, denselysampled rays may be projected from the center of mass in all directions.As graphically depicted, for example, in FIG. 22, each ray may intersectthe various contours. Processor 110 may be configured to compute themean and standard deviation of the obtained intersection locations foreach ray. If the standard deviation is below a threshold (which may meanthat the intersections are nearby and the contours are consistent alongthe ray), the mean point may be used as a high confidence location for acontour. High confidence locations from a plurality of rays may begrouped into high confidence segments, which may produce a contour thathas missing parts and multiple sub-segments. The missing segments may bereconstructed by examining the head shapes in the training images,selecting the head shape that is most consistent with the highconfidence segments, and copying the values from the selected head shapeinto the missing segments.

To detect a number of points in the face region of the imageinformation, processor 110 may be configured to perform a similar votingtechnique to that described above. For example, while the contourdescribed above is with respect to a head shape or face shape, the sameoperations may be performed for any other definable contour in the imageinformation of the subject. Processor 110 may be configured, forexample, to select points on one or more contours of the imageinformation (e.g., points surrounding the face, eyes, eyebrows, nose,mouth, and/or chin of the subject). For example, processor 110 may beconfigured to select a number of evenly spaced points along a givencontour.

To align the face region, processor 110 may be configured to perform oneor more of a translation, rotation, and scaling operation such that theresulting face is maximally aligned with an average face model. In someembodiments, regions of interest may be determined based on their knownassociation to corresponding facial regions determined based on thedetected points. The regions of interest may then be aligned to theaverage face model.

Processor 110 may be configured to perform a first evaluation of theexternal soft tissue image information using at least one of an anchoredcell analysis, a shifting patches analysis, and a relative measurementsanalysis (step 220). To perform the anchored cell analysis, processor110 may be configured to overlay a grid with a plurality of cells on theexternal soft tissue information, calculate descriptors for each of theplurality of cells, aggregate the descriptors to produce a vector, andcompare the vector to previously produced vectors from external softtissue images of other individuals previously diagnosed with the medicalcondition.

Overlaying the grid of cells may include a number of differentcomplementary and/or alternative options. For example, as depicted inFIG. 23A, processor 110 may be configured to overlay a fixed grid with aplurality of cells on the external soft tissue image information. Afixed grid of cells may include, for example, a plurality of adjacentsquares, rectangles, or triangles that are overlaid on a region (e.g., aface region) of the external soft tissue image information. As anotherexample, as depicted in FIG. 23B, processor 110 may be configured tooverlay a small grid of cells, which may be smaller in size than thecells of the fixed grid discussed above, on a particular defined region,such as at least one of a forehead region, a periorbital region, a nasalregion, a mid-face region, an ear region, and an oral region of theexternal soft tissue image information; processor 110 may be configuredto discount at least one other region of the external soft tissue imageinformation (e.g., a region from which minimal, or no, relevantinformation can be obtained, such as a hair region, may not be overlaidwith a small grid of cells). As another example, as depicted in FIG.23C, processor 110 may also be configured to overlay a plurality oftriangle cells generated by connecting points detected on the externalsoft tissue image information. For example, processor 110 may determinea plurality of feature points on the image information and connect thefeature points to form triangular regions.

To calculate descriptors for each of the plurality of cells, processor110 may be configured, for example, to analyze at least one of anintensity, a texture, and an edge associated with the external softtissue image information. In some embodiments, the descriptor for eachcell is a vector that includes, for example, data derived from at leastone of a SIFT, HOG, a self-similarity descriptor, a histogram of LocalBinary Patterns, and any other type of feature used in image analysisand computer vision. In some embodiments, the descriptor for each cellmay be a vector that includes, for example, data derived from at leastone layer in a neural network.

To aggregate the descriptors to produce a vector, processor 110 may beconfigured, for example, to create a vector that includes one or more ofthe calculated descriptors. For example, each of the descriptorsassociated with a cell in a fixed grid may be aggregated into a vector,each of the descriptors associated with a set of triangle cells may beaggregated into a vector, or each of the descriptors associated with asmall grid of cells may be aggregated into a vector. Additionally oralternatively, if more than one set of cells is created (e.g., both afixed grid and a set of triangle cells are formed, both a fixed grid anda small grid of cells are formed, both a set of triangle cells and afixed grid are formed, or a fixed grid, set of triangle cells, and smallgrid are all formed), a single vector may be created that includes thedescriptors of multiple sets of cells. Additionally or alternatively,more than one cell may be combined together to a plurality of additionaldescriptors, which also may be combined to form another level ofrepresentation, such as a plurality of layers in a neural network. Thevector that includes the aggregated descriptors may be referred to as,for example, a combined appearance vector.

To compare the vector to previously produced vectors from external softtissue images of other individuals previously diagnosed with the medicalcondition, processor 110 may be configured, for example, to access adatabase. The database may include, for example, vectors produced usingan anchored cell analysis of external soft tissue images of otherindividuals. The database may be annotated by one or more of patient ID,age, age group, gender, ethnic group, race group, dysmorphic feature,phenotypic feature, anthropometric measurements (including withoutlimitation, height, weight, head circumference, facial height, skullheight, upper facial height, lower facial height, head length, headwidth, facial width, mandibular width, anterior fontanelle size,posterior fontanelle size, inner canthal distance, outer canthaldistance, interpupillary distance, interorbital distance, palpebralfissure length, palpebral fissure height, obliquity of the palpebralfissure, orbital protrusion, corneal dimensions, ear length, ear width,ear protrusion, ear position, ear rotation, nasal height, length of thecolumella, nasal protrusion, nasal width, philtrum length, philtrumwidth, mouth width, ONC angle, maxillomandibular differential, andmandible width), relative ratios and proportions between bodily andfacial landmarks, known diagnosis, suspected diagnosis, mutations and/orgenetic variants, source of image, informed consent, pose, illumination,quality of image, expression type, and association with cohort (e.g.,part of a control group of individuals known not to be affected by amedical condition or part of a group of individuals known to be affectedby a medical condition). The database may also be annotated by, forexample, linking data regarding individuals in the database that arefamily members (e.g., siblings, parents, children, cousins, etc.) and/orindicating the relationship of other family members in the database thatare affected by a medical condition or dysmorphic feature to anindividual (e.g., sister of grandmother from the mother's side). In someembodiments, the previously produced vectors used in the comparison areassociated with one or more annotations that are in common. For example,the previously produced vectors may be associated with an annotationindicating that they were derived from images associated withindividuals of the same age, gender, and ethnicity as the subject.Additionally or alternatively, previously produced vectors used in thecomparison may be associated with a suspected dysmorphic feature and/ora suspected medical condition of the subject. That is, for example, thepreviously produced vectors may be associated with one or more ofindividuals affected by a dysmorphic feature, individuals in a controlgroup for the dysmorphic feature, individuals affected by a medicalcondition, and individuals in a control group for the medical condition.Additionally or alternatively, the previously described image cells maybe used as an input for a neural network to perform an association to anannotated database.

Data associated with a set of the most similar previously producedvectors (e.g., the 25 most similar previously produced vectors) may bedetermined. For example, processor 110 may be configured to calculateone or more of a Euclidean distance, a Chebyshev distance, a chi-squaredistance, a Mahalanobis distance, or other descriptor metrics, such asintersection kernel cosine similarity, or any other distance metric,including a metric that learned from a database to optimize the metricseparability between at least two classes between the combinedappearance vector and the previously produced vectors in the database todetermine a set of the most similar previously produced vectors (e.g.,the 25 previously produced vectors associated with the 25 shortestcomputed distances may be selected).

The set of the most similar previously produced vectors may be analyzed(at, for example, a server associated with the database or by processor110) to determine how many of the previously produced vectors areassociated with a positive example of a particular dysmorphic feature(that is, a previously produced vector associated with an individualknown to have the dysmorphic feature) and how many of the previouslyproduced vectors are associated with a negative example of a particulardysmorphic feature (that is, a previously produced vector associatedwith an individual known to not have the dysmorphic feature). Based onthe number of positive examples and the number of negative examples, aprobability score may be determined for the dysmorphic feature. Aprobability score may be calculated also based on the correspondence ofthe previously produced vectors fused into at least one statistical orother model, learned on the previous calculated database. Such a modelmay capture the relevant information needed to associate the test vectorto at least one class from the database. Association may be achievedusing a probability per class or an association to a specific classdirectly. A probability score, as used herein, may be an actualprobability or some value that is reflective of a probability. Forexample, a probability score may provide some indication of thelikelihood that a subject has a dysmorphic feature. For example, if onlypositive examples are included in the set of most similar previouslypresented vectors, a very high or maximum probability score may bedetermined (e.g., a probability score of 100). If only negative examplesare included, then a very low probability score may be determined (e.g.,a probability score of 1). If a mixture of positive and negativeexamples is included, then the probability score may reflect the numberof positive examples and the number of negative examples. Theprobability score may or may not be directly proportional to the numberof positive and negative examples. For example, in some embodiments, ifa threshold number of positive examples are obtained, then the same veryhigh or maximum probability score may be determined regardless of thenumber of positive examples or negative examples. Moreover, aprobability score is not necessarily a positive value. In someembodiments, the probability score may be a negative score. Moreover,all of the probability scores do not necessarily add up to 100%. In someembodiments, a probability score may be any real valued score that isapproximately monotonic with respect to the underlying probability of acertain medical condition or dysmorphology.

In some embodiments, a probability score for the dysmorphic feature mayalso, or alternatively, be calculated based on a degree of similarity ofthe combined appearance vector to a given one of the previously producedvectors. For example, if an equal number of positive and negativeexamples are retrieved, but the combined appearance vector is moresimilar to the previously produced vectors associated with positiveexamples than the previously produced vectors associated with negativeexamples, then the probability score may be relatively high. Incontrast, if an equal number of positive and negative examples areretrieved, but the combined appearance vector is more similar to thepreviously produced vectors associated with negative examples than thepreviously produced vectors associated with positive examples, then theprobability score may be relatively low.

In some embodiments, processor 110 may calculate more than oneprobability score for a given dysmorphic feature. For example, oneprobability score may be determined by treating all positive andnegative examples equally, and another probability score may bedetermined that considers the similarity of the combined appearancevector to the positive and negative samples. Moreover, probabilityscores for a plurality of different dysmorphic features may becalculated in the same or substantially the same way.

While the above anchored cell analysis description refers to dysmorphicfeatures, the same process may also, or alternatively, be performed todetermine probability scores for one or more medical conditions. Forexample, rather than determining the association of previously presentedvectors to dysmorphic features, processor 110 may determine whichmedical conditions are associated with the previously presented vectors.

To perform the shifting patches analysis, processor 110 may beconfigured to overlay a plurality of densely spaced or overlappingpatches on the external soft tissue image information, calculate adescriptor vector for each of the plurality of patches, and compare eachdescriptor vector to previously produced vectors from a similar regionin external soft tissue images of other individuals previouslydetermined to be affected by the medical condition.

As depicted in FIG. 24, to overlay patches on the external soft tissueimage information, processor 110 may be configured, for example, tooverlay multiple densely spaced or overlapping patches, which optionallymay be of varying sizes, onto a region of the image information (e.g., aface region). For example, a square patch of a first size may beoverlaid on the region of the image information at every possibleposition (e.g., the square patch may be shifted by one pixel in everydirection until the patch is overlaid in every possible position) or ata subset of the possible positions (e.g., the square patch may beshifted by ten pixels in every direction in every direction until thepatch is overlaid over the entire region of the image information). Insome embodiments, a square patch of one or more different sizes may alsobe overlaid on the region of the image information.

To calculate a descriptor vector for each of the plurality of patches,processor 110 may be configured to compute, for example, data derivedfrom at least one of a scale-invariant feature transform (SIFT), ahistogram of oriented gradients (HOG), a self-similarity descriptor, ahistogram of Local Binary Patterns, and any other type of feature usedin image analysis and computer vision. In some embodiments, thedescriptor for each cell is a vector that includes, for example, dataderived from a at least one layer in a neural network.

To compare each descriptor vector to previously produced vectors from asimilar region in external soft tissue images of other individualspreviously determined to be affected by the medical condition, processor110 may be configured to access a database. For example, the samedatabase discussed above with respect to the anchored cell analysis, ora similar database, may include previously produced vectors for patchesof images of individuals previously determined to be affected by themedical condition. As described above, the database may annotate thepreviously produced vectors with a variety of data such as, for example,one or more of patient ID, age, age group, gender, ethnic group, racegroup, dysmorphic feature, phenotypic feature, anthropometricmeasurements (including without limitation, height, weight, headcircumference, facial height, skull height, upper facial height, lowerfacial height, head length, head width, facial width, mandibular width,anterior fontanelle size, posterior fontanelle size, inner canthaldistance, outer canthal distance, interpupillary distance, interorbitaldistance, palpebral fissure length, palpebral fissure height, obliquityof the palpebral fissure, orbital protrusion, corneal dimensions, earlength, ear width, ear protrusion, ear position, ear rotation, nasalheight, length of the columella, nasal protrusion, nasal width, philtrumlength, philtrum width, mouth width, ONC angle, maxillomandibulardifferential, and mandible width), relative ratios and proportionsbetween bodily and facial landmarks, known diagnosis, suspecteddiagnosis, mutations and/or genetic variants, source of image, informedconsent, pose, illumination, quality of image, expression type, andassociation with cohort (e.g., part of a control group of individualsknown not to be affected by a medical condition or part of a group ofindividuals known to be affected by a medical condition). The databasemay also be annotated by, for example, linking data regardingindividuals in the database that are family members (e.g., siblings,parents, children, cousins, etc.) and/or indicating the relationship ofother family members in the database that are affected by a medicalcondition or dysmorphic feature to an individual (e.g., sister ofgrandmother from the mother's side).

Processor 110 may compare one or more of the descriptor vectorsassociated with the patches of the image information with the previouslyproduced vectors in the database. The comparison may occur at a serverassociated with the database, or may occur directly by processor 110using, for example, information retrieved from the database. Thepreviously produced vectors used in the comparison may be from a similarregion as the descriptor vector in external soft tissue images of otherindividuals previously determined to be affected by one or more medicalconditions and of other individuals in a control group. A similar regionmay include, for example, a patch in image information in the databasethat is the same, or substantially the same, distance from a center ofmass of a face in the same or substantially the same direction. Asimilar region may also include, for example, a patch in imageinformation in the database that is associated with a same organ or typeof region as a patch associated with a respective descriptor vectorassociated with the image information of the subject. For example, ifthe descriptor vector associated with a particular patch of the imageinformation of the subject is within an nose region, the descriptorvector may be compared to one or more descriptor vectors in the databasethat are also associated with a nose region. In some embodiments, onlypatches in the database that point to a center of mass location that isnot relatively far away from the center of mass of the face region ofthe image information and/or a center of mass of a particular organ ortype of region are considered. Additionally or alternatively, thepreviously produced vectors may be used as part of a neural network toperform an association to an annotated database.

In some embodiments, the descriptor vector may be compared only topreviously produced vectors that are associated with one or moreannotations that are in common. For example, the previously producedvectors used in the comparison may be associated with an annotationindicating that they were derived from images associated withindividuals of the same age, gender, and weight. Additionally oralternatively, the previously produced vectors used in the comparisonmay be associated with a suspected dysmorphic feature and/or a suspectedmedical condition. That is, for example, the previously produced vectorsmay be associated with one or more of individuals affected by adysmorphic feature, individuals in a control group for the dysmorphicfeature, individuals affected by a medical condition, and individuals ina control group for the medical condition.

Data associated with a set of the most similar previously producedvectors in the database (e.g., the 25 most similar previously producedvectors) may be determined. For example, processor 110 may be configuredto calculate one or more of a Euclidean distance, a Chebyshev distance,a chi-square distance, a Mahalanobis distance, or other descriptormetrics, such as intersection kernel cosine similarity, or any otherdistance metric, including a metric that learned from a database tooptimize the metric separability between at least two classes, betweeneach descriptor vector and the previously produced vectors in thedatabase to determine a set of the most similar previously producedvectors (e.g., the 25 previously produced vectors associated with the 25shortest computed distances may be selected). Data associated with theset of the most similar previously produced vectors for each descriptorvector may be determined. The data may include, for example, one or moredysmorphic features and/or one or more medical conditions associatedwith the set of the most similar previously produced vectors.

The set of the most similar previously produced vectors may be analyzed(at, for example, a server associated with the database or by processor110) to determine how many of the previously produced vectors areassociated with a positive example of a particular dysmorphic feature(that is, a previously produced vector associated with an individualknown to have the dysmorphic feature) and how many of the previouslyproduced vectors are associated with a negative example of a particulardysmorphic feature (that is, a previously produced vector associatedwith an individual known to not have the dysmorphic feature). Based onthe number of positive examples and the number of negative examples, aprobability score may be determined for the dysmorphic feature. Forexample, if only positive examples are retrieved, a very high or maximumprobability score may be determined (e.g., a probability score of 100).If, for example, only negative examples are retrieved, a very lowprobability score may be determined (e.g., a probability score of 1). Ifa mixture of positive and negative examples is determined then theprobability score may reflect the number of positive examples and thenumber of negative examples. However, the probability score may or maynot be directly proportional to the number of positive and negativeexamples. For example, if a threshold number of positive examples areobtained, then in some embodiments the same very high or maximumprobability score as if only positive examples were found may bedetermined A probability score may be calculated also based on thecorrespondence of the previously produced vectors fused into at leastone statistical or other model, learned on the previously calculateddatabase. Such a model may capture the relevant information needed toassociate the test vector to at least one class from the database.Association may be achieved using a probability per class or anassociation to a specific class directly.

In some embodiments, a probability score for the dysmorphic feature mayalso, or alternatively, be calculated based on a degree of similarity ofthe descriptor vector to a given one of the previously produced vectors.For example, if an equal number of positive and negative examples areretrieved, but the descriptor vector is more similar to the previouslyproduced vectors associated with positive examples than the previouslyproduced vectors associated with negative examples, then the probabilityscore may be relatively high. In contrast, if an equal number ofpositive and negative examples are retrieved, but the descriptor vectoris more similar to the previously produced vectors associated withnegative examples than the previously produced vectors associated withpositive examples, then the probability score may be relatively low.Thus, more than one probability score may be calculated for a givendysmorphic feature. Moreover, probability scores for a plurality ofdifferent dysmorphic features may be calculated in the same orsubstantially the same way.

In some embodiments, a probability score for a dysmorphic feature mayalso depend on a degree to which a center of mass associated with thepatch associated with the descriptor vector corresponds to a center ofmass associated with a patch associated with a particular previouslyproduced vector. For example, a center of mass of a face of the subjectmay be a first distance and direction from the patch associated with thedescriptor vector, and a center of mass of a face in a previouslypresented image may be a second distance and direction from a patchassociated with the particular previously produced vector. The dataassociated with the particular previously produced vector (e.g., whetherit comes from a positive or negative example of a dysmorphic feature)may have more or less significance on the probability score based on thedegree to which the two distances and directions correspond.

While the above shifting patch analysis description refers to dysmorphicfeatures, the same process may also, or alternatively, be performed todetermine probability scores for one or more medical conditions. Forexample, rather than determining the association of previously presentedvectors to dysmorphic features, processor 110 may determine whichmedical conditions are associated with the previously presented vectors.

To perform the relative measurements analysis, processor 110 may beconfigured to calculate a plurality of relative measurements between aplurality of locations within the external soft tissue imageinformation, aggregate the plurality of measurements to produce a vectorfor the plurality of measurements, and compare the vector to previouslyproduced vectors from external soft tissue images of other individualspreviously determined to be affected by the medical condition.

To calculate a plurality of relative measurements between a plurality oflocations within the external soft tissue image information, processor110 may be configured to detect a plurality of feature points in theexternal soft tissue image information. For example, as depicted in FIG.25, a plurality of points in a face region of the image information maybe detected, including, for example, one or more points surrounding eyeregions, eyebrow regions, a nose region, a mouth region, and a chinregion of the image information. These feature points may be detectedusing, for example, the operations described above.

Using the feature points, a plurality of relative measurements may becalculated. The plurality of relative measurements may include, forexample, one or more distances between feature points, angles formed bysets of feature points, sizes of areas formed by sets of feature points,shapes defined by sets of feature points, ratios established by sets ofdistances, angles, and sizes, and any other relative measurement thatmay be performed using the detected feature points. Other relativemeasurements may include, for example, any of the measurements disclosedin “Handbook of normal physical measurements,” 2nd edition, 2009, OxfordUniversity Press, to Hall et al., which is incorporated herein byreference in its entirety.

To aggregate the plurality of measurements to produce a vector for theplurality of measurements, processor 110 may be configured create avector that includes one or more of the calculated relativemeasurements. For example, each of the relative measurements may beaggregated into a single vector or each of the relative measurements ofa certain type (e.g., relative measurements relating to distancemeasurements) may be aggregated into a vector.

To compare the vector to previously produced vectors from external softtissue images of other individuals previously determined to be affectedby the medical condition, processor 110 may be configured to access adatabase. For example, the same database discussed above with respect tothe anchored cell analysis and the shifting patch analysis, or a similardatabase, may include previously produced vectors for relativemeasurements of individuals previously determined to be affected by themedical condition. As described above, the database may annotate thepreviously produced vectors with a variety of data such as, for example,one or more of patient ID, age, age group, gender, ethnic group, racegroup, dysmorphic feature, phenotypic feature, anthropometricmeasurements (including without limitation, height, weight, headcircumference, facial height, skull height, upper facial height, lowerfacial height, head length, head width, facial width, mandibular width,anterior fontanelle size, posterior fontanelle size, inner canthaldistance, outer canthal distance, interpupillary distance, interorbitaldistance, palpebral fissure length, palpebral fissure height, obliquityof the palpebral fissure, orbital protrusion, corneal dimensions, earlength, ear width, ear protrusion, ear position, ear rotation, nasalheight, length of the columella, nasal protrusion, nasal width, philtrumlength, philtrum width, mouth width, ONC angle, maxillomandibulardifferential, and mandible width), relative ratios and proportionsbetween bodily and facial landmarks, known diagnosis, suspecteddiagnosis, mutations and/or genetic variants, source of image, informedconsent, pose, illumination, quality of image, expression type, andassociation with cohort (e.g., part of a control group of individualsknown not to be affected by a medical condition or part of a group ofindividuals known to be affected by a medical condition). The databasemay also be annotated by, for example, linking data regardingindividuals in the database that are family members (e.g., siblings,parents, children, cousins, etc.) and/or indicating the relationship ofother family members in the database that are affected by a medicalcondition or dysmorphic feature to an individual (e.g., sister ofgrandmother from the mother's side).

Processor 110 may compare an aggregated vector of relative measurementswith the previously produced vectors in the database. The comparison mayoccur at a server associated with the database, or may occur directly byprocessor 110 using, for example, information retrieved from thedatabase.

In some embodiments, the previously produced vectors used in thecomparison may be associated with one or more annotations that are incommon. For example, the previously produced vectors used in thecomparison may be associated with an annotation indicating that theywere derived from images associated with individuals of the same age,gender, and weight. Additionally or alternatively, the previouslyproduced vectors used in the comparison may be associated with asuspected dysmorphic feature and/or a suspected medical condition. Thatis, for example, the previously produced vectors used in the comparisonmay be associated with one or more of individuals affected by adysmorphic feature, individuals in a control group for the dysmorphicfeature, individuals affected by a medical condition, and individuals ina control group for the medical condition.

Data associated with a set of the most similar previously producedvectors in the database (e.g., the 25 most similar previously producedvectors) may be determined for at least one aggregated vector ofrelative measurements. For example, processor 110 may be configured tocalculate one or more of a Euclidean distance, a Chebyshev distance, achi-square distance, a Mahalanobis distance, or other descriptormetrics, such as intersection kernel cosine similarity, or any otherdistance metric, including a metric that learned from a database tooptimize the metric separability between at least two classes betweenthe aggregated vector of relative measurements and the previouslyproduced vectors in the database to determine a set of the most similarpreviously produced vectors (e.g., the 25 previously produced vectorsassociated with the 25 shortest computed distances may be selected). Thedata associated with the set of the most similar previously producedvectors may include, for example, one or more dysmorphic features and/orone or more medical conditions associated with the set of the mostsimilar previously produced vectors. For example, for each aggregatedvector of relative measurements, one or more dysmorphic featuresassociated with a predefined number of the most similar previouslyproduced vectors may be determined Additionally or alternatively, foreach aggregated vector of relative measurements, one or more medicalconditions associated with a predefined number of the most similarpreviously produced vectors may be determined.

The set of the most similar previously produced vectors may be analyzed(at, for example, a server associated with the database or by processor110) to determine how many of the previously produced vectors areassociated with a positive example of a particular dysmorphic feature(that is, a previously produced vector associated with an individualknown to have the dysmorphic feature) and how many of the previouslyproduced vectors are associated with a negative example of a particulardysmorphic feature (that is, a previously produced vector associatedwith an individual known to not have the dysmorphic feature). Based onthe number of positive examples and the number of negative examples, aprobability score may be determined for the dysmorphic feature. Aprobability score may be calculated also based on the correspondence ofthe previously produced vectors fused into at least one statistical orother model, learned on the previously calculated database. Such a modelmay capture the relevant information needed to associate the test vectorto at least one class from the database. Association may be achievedusing a probability per class or an association to a specific classdirectly. For example, if only positive examples are retrieved, a veryhigh or maximum probability score may be determined (e.g., a probabilityscore of 100). If, for example, only negative examples are retrieved, avery low probability score may be determined (e.g., a probability scoreof 1). If a mixture of positive and negative examples is determined thenthe probability score may reflect the number of positive examples andthe number of negative examples. However, the probability score may ormay not be directly proportional to the number of positive and negativeexamples. For example, if a threshold number of positive examples areobtained, then in some embodiments the same very high or maximumprobability score as if only positive examples were found may bedetermined.

In some embodiments, a probability score for the dysmorphic feature mayalso, or alternatively, be calculated based on a degree of similarity ofthe aggregated vector of relative measurements to a given one of thepreviously produced vectors. For example, if an equal number of positiveand negative examples are retrieved, but the aggregated vector ofrelative measurements is more similar to the previously produced vectorsassociated with positive examples than the previously produced vectorsassociated with negative examples, then the probability score may berelatively high. In contrast, if an equal number of positive andnegative examples are retrieved, but the aggregated vector of relativemeasurements is more similar to the previously produced vectorsassociated with negative examples than the previously produced vectorsassociated with positive examples, then the probability score may berelatively low. Thus, more than one probability score may be calculatedfor a given dysmorphic feature. Moreover, probability scores for aplurality of different dysmorphic features may be calculated in the sameor substantially the same way.

In some embodiments, the comparison to previously produced vectors maynot be a direct comparison. For example, the previously produced vectorsmay be analyzed to determine percentiles regarding various relativemeasurements in a population. Processor 110 may determine where variousrelative measurements in the aggregated vector of relative measurementsfall in a particular population. Processor 110 may, for example,determine the percentile of a particular dysmorphic feature of thesubject in a population (e.g., the length of a facial feature iscompared to a population). The population may be a general population ormay be some subset defined by, for example, an aspect of the subject(e.g., the subject's age, gender, ethnicity, etc.) Based on thepercentile, processor 110 may determine whether the subject is likely toexhibit a dysmorphic feature and determine a probability score of thedysmorphic feature. Processor 110 may also, or alternatively, beconfigured to determine a severity score associated with a dysmorphicfeature. For example, if processor 110 determines that the subject islikely to exhibit a dysmorphic feature, a determination may be made asto a severity score based on the determined percentile associated withthe subject.

As another example of an indirect comparison, in some embodiments, oneor more dysmorphic features may be defined directly by one or morerelative measurements. For example, an analysis of the previouslyproduced vector may demonstrate that a triangular face dysmorphicfeature or an up-slanting eye dysmorphic feature may be defined by oneor more angles or ranges of angles defined by a set of feature points.Thus, processor 110 may, for example, compare an aggregated vector ofrelative measurements to a defined dysmorphic feature. A probabilityscore may be determined based on whether or not the aggregated vector ofrelative measurements satisfies the defined dysmorphic feature and/or adegree to which the aggregated vector of relative measurements satisfiesthe defined dysmorphic feature. Processor 110 may also, oralternatively, be configured to determine a severity score associatedwith a dysmorphic feature. For example, if processor 110 determines thatthe subject is likely to exhibit a dysmorphic feature, a determinationmay be made as to a severity score based on the degree to which theaggregated vector of relative measurements satisfies the defineddysmorphic feature. The probability score and/or the severity scoredetermined using the relative measurements may be determined based on anormalization procedure. For example, in some embodiments, the lengthsof mouth-related measurements may be normalized based on the width ofthe face. The normalized measurements may then be analyzed using thedefined dysmorphic features.

While the above relative measurement analysis description refers todysmorphic features, the same process may also, or alternatively, beperformed to determine probability scores for one or more medicalconditions. For example, rather than determining the association ofpreviously produced vectors to dysmorphic features, processor 110 maydetermine which medical conditions are associated with the previouslyproduced vectors.

FIG. 26 depicts an example of each of the three techniques that may beused for relative measurements analysis. For example, the geometricposition of the feature points may define a square face dysmorphicfeature. The location on a philtrum length distribution curve maysuggest that the subject has a short philtrum. The aggregated vector ofrelative measurements may be most similar to two positive examples of ashort nose and one negative example of a short nose.

Processor 110 may be configured to generate first evaluation resultinformation based, at least in part, on the first evaluation (step 230).Processor 110 may utilize the data derived from the first evaluation togenerate the first evaluation result. For example, as discussed above,one or more probability scores for one or more dysmorphic featuresand/or one or more medical conditions may be determined in the firstevaluation. If a plurality of probability scores for any particulardysmorphic feature and/or any particular medical condition aredetermined in the first evaluation, then the probability scores for theparticular dysmorphic feature and/or the particular medical conditionmay be combined. As one example, an average of the probability scoresmay be determined. As another example, the plurality of probabilityscores for a particular dysmorphic feature and/or a particular medicalcondition may be input into a classifier that is calibrated to outputanother probability score attributed to the particular dysmorphicfeature and/or the particular medical condition. For example, theclassifier may be trained with positive and negative examples of amedical condition in order to determine a single probability scoreand/or severity score of a particular dysmorphic feature and/or aparticular medical condition based on a received set of probabilityscores. In some embodiments, the classifier may also, or alternatively,be configured to receive a set of severity scores in order to determinea probability score for the medical condition.

Processor 110 may be configured to perform a second evaluation of theexternal soft tissue image information using at least one of theanchored cell analysis, the shifting patches analysis, and the relativemeasurements analysis (step 240). For example, if the first evaluationincludes an anchored cell analysis, then a shifting patches analysis orrelative measurements analysis may be performed as the secondevaluation. If the first evaluation includes a shifting patchesanalysis, then an anchored cell analysis or relative measurementsanalysis may be performed. If the first evaluation includes a relativemeasurements analysis, then an anchored cell analysis or shiftingpatches analysis may be performed as the second evaluation. The anchoredcell analysis, shifting patches analysis, and relative measurementsanalysis may be performed in the same or substantially the same manneras described above with respect to step 220.

In some embodiments, the first evaluation and second evaluation may beof the same general type (e.g., both may be an anchored cell analysis,both may be a shifting patches analysis, or both may be relativemeasurements analysis). In such embodiments, one of the evaluations mayresult in, for example, one or more probability scores associated withone or more dysmorphic features, whereas another one of the evaluationsmay result in, for example, one or more probability scores associatedwith one or more medical conditions. Likewise, even if the general typesof analysis are different, one of the evaluations may result in, forexample, one or more probability scores associated with one or moredysmorphic features, whereas another one of the evaluations may resultin, for example, one or more probability scores associated with one ormore medical conditions.

Processor 110 may be configured to generate second evaluation resultinformation based, at least in part, on the second evaluation (step250). Processor 110 may utilize the data derived from the secondevaluation to generate the second evaluation result. For example, one ormore probability scores and/or severity scores for one or moredysmorphic features and/or one or more medical conditions may bedetermined in the second evaluation that are combined using one or moreclassifiers to generate a single probability score associated with oneor more particular dysmorphic features and/or medical conditions.

Processor 110 may be configured to predict a likelihood that the subjectis affected by the medical condition based, at least in part, on thefirst evaluation result information and the second evaluation resultinformation (step 260). For example, if the first evaluation resultinformation includes one or more probability scores associated with oneor more dysmorphic features, and the second evaluation resultinformation includes one or more probability scores associated with oneor more dysmorphic features, processor 110 may be configured to analyzethe information to determine the likelihood. Alternatively, for example,if the first evaluation result information includes one or moreprobability scores associated with one or more medical conditions, andthe second evaluation result information includes one or moreprobability scores associated with one or more medical conditions,processor 110 may be configured to analyze the information to determinethe likelihood. Alternatively, for example, if the first evaluationresult information includes one or more probability scores associatedwith one or more medical conditions, and the second evaluation resultinformation includes one or more probability scores associated with oneor more dysmorphic features, processor 110 may be configured to analyzethe information to determine the likelihood. Likewise, if, for example,the first evaluation result information includes one or more probabilityscores associated with one or more dysmorphic features, and the secondevaluation result information includes one or more probability scoresassociated with one or more medical conditions, processor 110 may beconfigured to analyze the information to determine the likelihood.

If both evaluations return a set of probability scores for a set ofdysmorphic features, then the set of probability scores for the set ofdysmorphic features may be input into a trained classifier that iscalibrated to output a probability score attributed to a particularmedical condition. For example, the classifier may be trained withpositive and negative examples of the medical condition in order todetermine a probability score of the particular medical condition.

If both evaluations return a set of probability scores for a set ofmedical conditions, then the set of probability scores for a particularmedical condition may be input into a trained classifier that iscalibrated to output another probability score attributed to theparticular medical condition. For example, the classifier may be trainedwith positive and negative examples in order to determine a probabilityscore of the particular medical condition. In this way, a more accuratemedical condition probability score may be determined than any one ofthe individual evaluations that produced a medical condition likelihood.

If one of the evaluations returns a set of probability scores for a setof dysmorphic features and another one of the evaluations results a setof probability scores for a medical condition, then the set ofprobability scores for the set of dysmorphic features from oneevaluation and the set of probability scores for a particular medicalcondition from the other evaluation, may be input into a trainedclassifier that is calibrated to output another probability scoreattributed to a particular medical condition. For example, theclassifier may be trained with positive and negative examples of amedical condition in order to determine a probability score of theparticular medical condition.

In some embodiments an initial determination of the likelihood that thesubject is affected by the medical condition may be revised. Forexample, a likelihood of a plurality of medical conditions may initiallybe determined. Then, a revised likelihood of one or more of the medicalconditions may be determined based on the other likelihoods. Forexample, if two medical conditions are related such that they typicallyoccur together, a low probability score for one medical condition maydecrease an otherwise high probability score for the other medicalcondition. Similarly, if two medical conditions are related such thatthey typically do not occur together, a high probability score for bothmedical conditions may cause both probability scores to decrease.Moreover, for example, if a set of medical conditions are initiallydetermined to have a high probability score, but are known to commonlybe misdiagnosed for another medical condition, then the probabilityscore of each medical condition in the set may be decreased and theprobability score of the other medical condition may be increased.

Processor 110 may be configured to consider other data as well whendetermining the likelihood of the medical condition. For example, morethan two evaluations may be performed in substantially the same manneras the first and second evaluations described above. Processor 110 maybe configured to analyze the additional evaluations using the techniquesdescribed above. Moreover, features associated with the subject may bedetermined from other sources. For example, a physician may provide(e.g., dictate or type) one or more known features (e.g., dysmorphicfeatures, biographical information, demographic information, etc.) ofthe subject that are used, for example, to limit the images in thedatabase that the subject is compared to (e.g., the subject may only becompared against other individuals sharing one or more of the featuresprovided by the physician).

FIG. 3 illustrates an exemplary process 300 that processor 110 may beconfigured to perform. For example, as discussed above, processor 110may be configured to perform process 300 by executing software orfirmware stored in memory device 120, or may be configured to performprocess 300 using dedicated hardware or one or more ASICs.

Processor 110 may be configured to receive information reflective of anexternal soft tissue image of the subject (step 310). Processor 110 maybe configured, for example, to perform step 310 in the same manner asstep 210, discussed above.

Processor 110 may be configured to divide the external soft tissue imageinformation into a plurality of regions (step 320). For example,processor 110 may be configured to process at least one of a foreheadregion of the external soft tissue image information, a periorbitalregion of the external soft tissue image information, a nasal region ofthe external soft tissue image information, a mid-face region of theexternal soft tissue image information, an ear region of the externalsoft tissue image information, and an oral region of the external softtissue image information, and discount at least one other region of theexternal soft tissue image information. Particular regions in theexternal soft tissue image information may be defined in accordancewith, for example, any of the techniques described above. For example,as depicted in FIG. 23B, a small grid may be applied to a nasal regionof the external soft tissue image information. Areas surrounding thenasal region may be discounted by, for example, not having a small gridapplied to them.

In addition, while regions associated with a face of the subject arediscussed, other regions may also be processed. For example, theexternal soft tissue image information may also, or alternatively,include a side view of the subject that includes an ear region. FIG. 27depicts an example of an ear region that is further divided into aplurality of regions.

Processor 110 may be configured to generate an analysis of each of theplurality of regions (step 330). For example, within each region, atleast one of an anchored cell analysis and a shifting patch analysis maybe performed. The anchored cell analysis and shifting patch analysis maybe performed in the manner described above with respect to step 220. Asdescribed above, processor 110 may be configured to calculate adescriptor for each of the plurality of regions. The descriptor mayinclude, for example, at least one of a combined appearance vector if ananchored cell analysis is performed and a descriptor vector if ashifting patch analysis is performed.

As described above, processor 110 may be configured to compare theplurality of regions with data derived from images of individuals knownto be affected by the medical condition. For example, as describedabove, processor 110 may be configured to compare the descriptor topreviously produced descriptors from additional external soft tissueimages of other individuals previously determined to be affected by themedical condition. Based on the comparison, for each region, one or moreprobability scores associated with one or more dysmorphic featuresand/or one or more medical conditions may be determined.

Processor 110 may further be configured to aggregate the analyses of theplurality of regions (step 340). For example, an analysis of a nasalregion may result in a first set of probability scores regardingdysmorphic features and/or medical conditions associated with the nasalregion and an analysis of an ear region may result in a second set ofprobability scores regarding dysmorphic features and/or medicalconditions associated with the ear region. The probability scores may beproduced using, for example, the techniques described above. In someembodiments, some of the probability scores may be aggregated bycombining the probability scores. For example, a particular dysmorphicfeature and/or medical condition may be associated with both the nasalregion and the ear region. A probability score for the particulardysmorphic feature and/or medical condition determined from the nasalregion and another probability score for the particular dysmorphicfeature and/or medical condition determined from the ear region may beinput into a classifier trained using, for example, positive andnegative examples of the particular dysmorphic feature and/or medicalcondition, to generate a third probability score reflective of aprobability score associated with the particular dysmorphic featureand/or medical condition. As a result of the aggregation, a singleprobability score, for example, may be determined for a set ofdysmorphic features and/or medical conditions.

Processor 110 may further be configured to determine a likelihood thatthe subject is affected by the medical condition based on the aggregatedanalyses (step 350). For example, in the same or substantially the samemanner as described above with respect to step 260, one or moreclassifiers may be trained to receive a set of scores for a plurality ofdysmorphic features and/or medical conditions and output a scorerepresentative of the likelihood that the subject is affected by themedical condition.

In some embodiments, the medical condition may be a known medicalcondition. However, some medical conditions can have unknown geneticcauses. Processor 110 may be configured to identify an underlyinggenetic variation likely to cause a medical condition. For example, adatabase, such as the database discussed above, may include a pluralityof external soft tissue images of individuals associated with a medicalcondition caused by an unknown genetic variation. The database may alsoinclude a plurality of genetic variation information of individualshaving a medical condition caused by an unknown genetic variation.Processor 110 may be further configured to determine whether a commondysmorphology exists at the location of at least some of the pluralityof external soft tissue images. For example, processor 110 may beconfigured to analyze the plurality of external soft tissue images inthe manner described above to determine one or more dysmorphic features.A common dysmorphology may exist if, for example, a dysmorphic featureexists at the same or substantially the same location in at least twoimages.

Processor 110 may be further configured to analyze the plurality ofgenetic variation information to identify at least one common geneticvariation. A common genetic variation may include, for example, adetermination that a gene associated with one image matches a geneassociated with another image. Processor 110 may be further configuredto compare the location of the plurality of external soft tissue imageswith the common genetic variation. For example, a determination may bemade whether the genetic variation is known to affect a body partcontaining the common dysmorphology location in the images. Processor110 may be further configured to associate, in the database, at leastone common location in the plurality of external soft tissue images andat least one common genetic variation in the plurality of geneticvariation information.

FIG. 4 illustrates an exemplary process 400 that at least one processormay be configured to perform. For example, as discussed above, processor110 may be configured to perform process 400 by executing software orfirmware stored in memory device 120, or may be configured to performprocess 400 using dedicated hardware or one or more ASICs.

Processor 110 may be configured to receive information reflective of anexternal soft tissue image of the subject (step 410). Step 410 mayinclude substantially the same operations as step 210, discussed above.

Processor 110 may be further configured to analyze the external softtissue image information of the subject to generate medical-conditionrelated information (step 420). For example, medical-condition relatedinformation may be generated using the same or substantially the sameoperations described above with respect to steps 220-260. Optionally,however, only one analysis may be performed (e.g., the analysis in steps220-230) rather than the two (or more) analyses described in steps220-250. Medical-condition related information may include, for example,one or more scores for one or more dysmorphic features and/or one ormore medical conditions.

Processor 110 may be further configured to analyze external soft tissueimage information of at least one relative of the subject known not tobe affected by the medical condition to generate additionalmedical-condition related information (step 430). For example, processor110 may be configured to determine one or more scores associated withone or more dysmorphic features associated with the relative. Processor110 may be configured to identify, for example, dysmorphic featureshaving a high score that are usually indicative of a medical conditionthat the relative is known not to be affected by. In some embodiments,external soft tissue image information of a plurality of relatives isanalyzed and dysmorphic features are identified having a high score forall of the relatives, or a number of relatives greater than a threshold,that are usually indicative of a medical condition that the relativesare known not to be affected by.

Additionally, or alternatively, processor 110 may be configured todetermine one or more scores associated with one or more medicalconditions associated with the relative. For example, processor 110 mayinitially utilize a classifier used for the general population todetermine a score associated with a medical condition that the relativeis known not to have. However, despite the relative not having themedical condition, processor 110 may determine a high score for themedical condition based on the analysis of the image informationassociated with the relative.

Processor 110 may be further configured to predict a likelihood that thesubject is affected by a medical condition by discounting features inthe medical-condition information that are common to the additionalmedical-condition related information (step 440). For example, asdescribed above, processor 110 may determine one or more scores of oneor more dysmorphic features from the image information of the subject.Based on the identified dysmorphic features of one or more relatives(e.g., the dysmorphic features of the relative that exhibit a high scoreusually indicative of a medical condition despite the relative beingknown not to be affected by the medical condition), processor 110 may beconfigured to modify or construct a classifier associated with aparticular medical condition. For example, if a high score for aparticular dysmorphic feature is usually used to increase theprobability of a medical condition, but the relative has a high scorefor the particular dysmorphic feature and is known not to have themedical condition, the classifier may be modified or constructed suchthat the dysmorphic feature is ignored, is used to reduce the likelihoodof the medical condition, or is used less heavily than for the generalpopulation in the determination of the likelihood of the medicalcondition. As another example, if processor 110 determines a high scorefor a medical condition for a relative of the subject despite therelative being known not to be affected by the medical condition,processor 110 may reduce any score determined for the medical conditionfor the subject. As another example, one or more images of the relativemay be used to train the classifier. For example, one or more images ofone or more relatives known not to be affected by a medical conditionmay be used a negative examples when training a classifier. As anotherexample, only dysmorphic features of the subject that are different thanone or more dysmorphic features of one or more relatives known not to beaffected by a medical condition may be used in the likelihooddetermination.

FIG. 5 illustrates an exemplary process 500 that at least one processormay be configured to perform. For example, as discussed above, processor110 may be configured to perform process 500 by executing software orfirmware stored in memory device 120, or may be configured to performprocess 500 using dedicated hardware or one or more ASICs.

Processor 110 may be configured to receive first electronic informationreflective of an external soft tissue image of the subject (step 510).Processor 110 may be configured, for example, to perform step 510 in thesame manner as step 210, discussed above. Further, the receivedinformation may be first electronic information reflective of first setsof values corresponding to pixels of a cranio-facial external softtissue image of the subject. The first sets of values may correspond torelationships between at least one group of pixels in the cranio-facialsoft tissue image of the subject. For example, the first sets of valuesmay be pixel values corresponding to a region of interest in thecranio-facial external soft tissue image. The region of interest may bethe entire cranio-facial external soft tissue image or a subset thereof,depending on implementation-specific considerations. For example, insome embodiments, only a predetermined section of the acquiredcranio-facial external soft tissue image may be of interest because thepredetermined section corresponds to a portion of the subject beingscreened for a symptom of a genetic condition. The relationship betweenthe pixels in the at least one group of pixels may define, for instance,the spatial relationship between the pixels in the at least one group ofpixels, the relative values of the pixels in the at least one group ofpixels, or any other informative relationship.

Processor 110 may be configured to access second electronic informationreflective of external soft tissue images of a plurality ofgeographically disbursed individuals (step 515). The second electronicinformation may be further reflective of second sets of valuescorresponding to pixels of cranio-facial external soft tissue images ofthe plurality of geographically dispersed individuals. The second setsof values may correspond to pixel values of associated pixels in digitalimages of the plurality of geographically dispersed individuals. As usedherein, geographically dispersed individuals may be geographicallydispersed, for example, in different countries, states, counties,cities, etc. Indeed, geographical dispersion encompasses individualslocated in a variety of types of geographies, such as different physicallocations, climates, elevations, etc.

Processor 110 may be configured to use image information analysis toprocess the first electronic information with respect to the secondelectronic information (step 520). For example, processor 110 may beconfigured to use image information analysis, wherein the imageinformation analysis includes at least one of anchored cells analysis,shifting patches analysis, and relative measurements analysis. Theanchored cell analysis, shifting patches analysis, and relativemeasurements analysis may be performed in the same or substantially thesame manner as described above. As described above (for example, in step220), processor 110 may analyze the external soft tissue imageinformation based on a plurality of objective criteria, including atleast one of age, gender, and ethnicity. For example, the only externalsoft tissue images in the database of other subjects of the same age,gender, and ethnicity as the subject may be utilized in the analysis.

Processor 110 may be further configured to determine, based on the imageinformation analysis, dysmorphic features included in the external softtissue image information (step 530). For example, as described above,one or more of an anchored cells analysis, shifting patches analysis,and relative measurements analysis may be used to assign a probabilityscore to each dysmorphic feature in a set of dysmorphic features beinganalyzed.

Processor 110 may be further configured to access descriptors associatedwith the dysmorphic features, for example, from a suitable database ofsuch descriptors (step 540). In some embodiments, the accesseddescriptors include a list of words associated with dysmorphic featuresand that are potential indicators of at least one medical condition. Forexample, the accessed descriptors may include terms that are compatiblewith a plurality of databases, such as medical ontology databases, forsearching medical conditions. The descriptors associated with thedysmorphic features may be obtained, for example, from a variety ofsources including, for example, the International StatisticalClassification of Diseases and Related Health Problems (e.g., ICD-9 orICD-10), the Human Phenotype Ontology (HPO), and various other sourcesof descriptions for dysmorphic features, such as medical books,published journal papers, and computerized datasets. Processor 110 maybe configured to link the descriptors associated with the dysmorphicfeatures obtained from different sources (e.g., a descriptor for aparticular dysmorphic feature used in ICD-10 may be linked to adescriptor for the particular dysmorphic feature used in HPO). Eachdescriptor for a dysmorphic feature may include, for example, atextugeal description and a list of synonyms. In some embodiments,HPO-based descriptors may be used as a reference list and all otherlists from other sources may be mapped to it. If there is a termdysmorphic feature that is missing from HPO, it may be given a uniqueHPO-like numeric identifier and added to the reference list. In someembodiments, processor 110 may determine when HPO is updated and, basedon a determined that HPO has updated, update the reference list.Moreover, in some embodiments, the accessed descriptors are a list ofwords that includes at least one description of a general appearance ofa medical condition.

As one example, six high scoring dysmorphic features may be determinedfor image information of a subject. The descriptors for the dysmorphicfeatures may include, for example, “Vermillion, Lower Lip, Thick”,“Columella, High Insertion”, “Hairline, High Anterior or Forehead,Tall”, “Palpebral Fissure, Upslanted”, “Eyebrow, Thick or Hypertrichosisof the Eyebrow or Bushy Eyebrow”, “and “Philtrum, Tented.” Thus, each ofthe descriptors may include, for example, a name of the dysmorphicfeature (e.g., “Eyebrow, Thick”) and possible alternatives to thedysmorphic feature (e.g., “Hypertrichosis of the Eyebrow”).

Processor 110 may be further configured to select and/or output one ormore of the descriptors (step 550). For example, processor 110 may beconfigured to output at least some of the descriptors to output device150, for example, by reconfiguring the output device 150 to display acranio-facial image of the subject together with at least one descriptorand an indication in the image of the location of the at least onedysmorphic feature. Output device 150 may be, for example, a display. Insome embodiments, as depicted in FIG. 1, output device 150 may be partof system 100. However, in other embodiments, output device 150 may belocated remotely and processor 110 may be configured to send data to adevice that includes output device 150 or is in communication withoutput device 150. A display may include, for example, one or more of atelevision set, computer monitor, head-mounted display, broadcastreference monitor, a liquid crystal display (LCD) screen, alight-emitting diode (LED) based display, an LED-backlit LCD display, acathode ray tube (CRT) display, an electroluminescent (ELD) display, anelectronic paper/ink display, a plasma display panel, an organiclight-emitting diode (OLED) display, thin-film transistor display (TFT),High-Performance Addressing display (HPA), a surface-conductionelectron-emitter display, a quantum dot display, an interferometricmodulator display, a swept-volume display, a carbon nanotube display, avariforcal mirror display, an emissive volume display, a laser display,a holographic display, a light field display, a projector and surfaceupon which images are projected, a printer configured to generate aprintout of data, or any other electronic device for outputting visualinformation.

Output device 150 may also be an audio device configured to output audiorepresentative of, for example, at least some of the descriptors. Theaudio device may include, for example, a sound card and one or morespeakers. Processor 110 may be configured, for example, to convert atleast some of the descriptors into audio using a text-to-speech program.

In some embodiments, the descriptors may be presented in a list. In someembodiments, an image may be displayed near a descriptor that isindicative of the general location of the dysmorphic feature associatedwith the descriptor. For example, an image of an eye may be displayednext to a descriptor of “Eyebrow, Thick”.

In some embodiments, a descriptor may be displayed at a location at, orclose to, a dysmorphic feature to which it is associated. For example,the image information of the subject may be presented on the display. Adescriptor (e.g., “Eyebrow, Thick”) may be displayed on top of a regionof the image information associated with a dysmorphic feature associatedwith the descriptor (e.g., “Eyebrow, Thick” may be displayed on top ofan eye or eyebrow region of the image information).

FIG. 6 illustrates an exemplary process 600 that at least one processormay be configured to perform. For example, as discussed above, processor110 may be configured to perform process 600 by executing software orfirmware stored in memory device 120, or may be configured to performprocess 600 using dedicated hardware or one or more ASICs.

Processor 110 may be configured to receive information reflective of anexternal soft tissue image of the subject (step 610). Processor 110 maybe configured, for example, to perform step 610 in the same manner asstep 210 discussed above.

Processor 110 may be configured to define at least one hundred locationsin the received external soft tissue image information, whereininformation about the at least one hundred locations constitutes subjectinformation (step 620). Processor 110 may be configured to define the atleast one hundred locations by determining at least one hundred featurepoints in the manner described above for determining feature points.

Processor 110 may also be configured to receive first informationdefining at least one hundred locations in at least an external softtissue image of at least a first individual known to be affected by themedical condition (step 630) and to receive second information definingat least one hundred locations in at least an external soft tissue imageof at least a second individual known to be affected by the medicalcondition (step 640). The subject information, the first information,and the second information may include, for example, vector data, ratiodata, distance data, angular data, area data, and shape data associatedwith a relative measurements analysis calculated between at least someof the at least one hundred locations.

Processor 110 may be configured to determine a likelihood that thesubject is affected by a medical condition by comparing the subjectinformation with the first information and the second information (step650). For example, processor 110 may be configured to determine alikelihood that the subject is affected by a medical condition bycomparing the subject information with the first information and thesecond information using the relative measurements analysis describedabove.

In some embodiments, processor 110 may initially define a first numberof feature points (e.g., one hundred feature points). The first numberof feature points may permit processor 110 to determine the likelihoodat a first speed. However, if the likelihood determination isinconclusive result (e.g., the likelihood is above a first threshold butbelow a second threshold), the above process may be repeated with asecond number of feature points greater than the first number of featurepoints (e.g., one thousand feature points) that requires more time, butmay be more accurate.

FIG. 7 illustrates an exemplary process 700 that at least one processormay be configured to perform. For example, as discussed above, processor110 may be configured to perform process 700 by executing software orfirmware stored in memory device 120, or may be configured to performprocess 700 using dedicated hardware or one or more ASICs.

Processor 110 may be configured to receive information reflective of anexternal soft tissue image of the subject (step 710). Processor 110 maybe configured, for example, to perform step 710 in the same manner asstep 210 discussed above.

Processor 110 may be configured to analyze the external soft tissueimage information to identify locations likely to be associated with atleast one dysmorphology corresponding to a medical condition (step 720).For example, processor 110 may be configured to identify one or moredysmorphic features having high probability scores in the mannerdescribed above.

Processor 110 may be configured to superimpose indicators of the atleast one dysmorphology on the external soft tissue image information(step 730). For example, processor 110 may be configured to output to adisplay the external soft tissue image information along with asuperimposed indication of at least one dysmorphology. For example,points detected in the image information may be superimposed on theimage information. As another example, regions in the image informationassociated with high probability dysmorphic features may be highlighted.As another example, a heat map may be superimposed on the imageinformation such that, at locations in the external soft tissue imageinformation associated with a dysmorphic feature having a low score, afirst translucent color may be used, whereas at locations in theexternal soft tissue image information associated with a dysmorphicfeature having a high score, a second translucent color, different thanthe first translucent color, may be used. The locations may be chosen,for example, based on cells or patches used to determine the presence ofthe dysmorphic feature. In some embodiments, processor 110 may beconfigured to blur the heat map to produce a more appealing heat map.

FIG. 8 illustrates an exemplary process 800 that at least one processormay be configured to perform. For example, as discussed above, processor110 may be configured to perform process 800 by executing software orfirmware stored in memory device 120, or may be configured to performprocess 800 using dedicated hardware or one or more ASICs.

Processor 110 may be configured to receive information reflective of anexternal soft tissue image of the subject (step 810). Processor 110 maybe configured, for example, to perform step 810 in the same manner asstep 210 discussed above.

Processor 110 may be configured to display the external soft tissueimage information (step 820). For example, processor 110 may beconfigured to output the external soft tissue image information to adisplay that is configured in the manner described above.

Processor 110 may be configured to enable a user to select a region ofthe external soft tissue image information (step 830). For example,processor 110 may be configured to enable a user to select a region ofthe external soft tissue image information presented on the display. Insome embodiments, the region selected by the user may be expanded afterprocessor 110 detects the selection.

Processor 110 may be configured to identify, to the user, informationabout dysmorphic features in the selected region (step 840). Forexample, information about the dysmorphic features may be displayed in alist or may be superimposed onto the external soft tissue imageinformation. The list of dysmorphic features may be presented, forexample, in descending or ascending order of probability score.

FIG. 9 illustrates an exemplary process 900 that at least one processormay be configured to perform. For example, as discussed above, processor110 may be configured to perform process 900 by executing software orfirmware stored in memory device 120, or may be configured to performprocess 900 using dedicated hardware or one or more ASICs.

Processor 110 may be configured to receive information reflective of anexternal soft tissue image of the subject (step 910). Processor 110 maybe configured, for example, to perform step 910 in the same manner asstep 210 discussed above.

Processor 110 may be configured to analyze the external soft tissueimage information (step 920). For example, processor 110 may analyze theexternal soft tissue image information using the same or substantiallythe same operations described above with respect to steps 220-260.Optionally, however, only one analysis may be performed (e.g., theanalysis in steps 220-230) rather than the two (or more) analysesdescribed in steps 220-250. For example, as described above, processor110 may be configured to perform at least one of at least one ofanchored cells analysis, a shifting patches analysis, and a relativemeasurements analysis.

Processor 110 may be configured to identify one or more external softtissue attributes in the external soft tissue image information based,at least in part, on the analysis (step 930). The one or more externalsoft tissue attributes may include, for example, one or more dysmorphicfeatures. For example, as described above, processor 110 may beconfigured to identify potential external soft tissue attributes in theexternal soft tissue image information and to assign a confidence levelto the potential external soft tissue attributes reflective of alikelihood that the potential external soft tissue attributes appear inthe image. In some embodiments, processor 110 may be configured toidentify which external soft tissue attributes are indicators of medicalconditions by taking into account a weighting of each external softtissue attribute as an indicator of each medical condition. For example,processor 110 may identify all dysmorphic features having a highprobability score or a probability score above a predeterminedthreshold. In some embodiments, the weighting of each external softtissue attribute includes at least one of a severity of each externalsoft tissue attribute, a commonality of each external soft tissueattribute in a general population, and a relevance of each external softtissue attribute to a medical condition.

Processor 110 may be configured to access at least one database ofexternal soft tissue attributes associated with a plurality of medicalconditions (step 940). For example, processor 110 may be configured toaccess a database containing data regarding one or more dysmorphicfeatures and/or one or more medical conditions in the same manner asdescribed above in, for example, step 220.

Processor 110 may be configured to compare the one or more identifiedexternal soft tissue attributes with the external soft tissue attributesof the at least one database (step 950). For example, processor 110 maybe configured to compare the one or more identified external soft tissueattributes with the external soft tissue attributes of the at least onedatabase in the same manner as described above in, for example, step220. The comparison may generate one or more probability scoresassociated with one or more dysmorphic features.

Processor 110 may be configured to output information about at least onemedical condition likely possessed by the subject based on thecomparison (step 960). For example, as described above, processor 110may be configured to input the one or more probability scores of the oneor more dysmorphic features into a classifier to generate a probabilityscore for a medical condition. In some embodiments, processor 110 isconfigured to determine additional information about the at least onemedical condition likely possessed by the subject based directly on theanalysis; and output the information about at least one medicalcondition likely possessed by the subject based on the comparison andthe additional information. For example, as described in step 260,processor 110 may determine a likelihood of a medical condition based onboth an initial medical condition likelihood and a set of dysmorphicfeature probability scores.

FIG. 10 illustrates an exemplary process 1000 that at least oneprocessor may be configured to perform. For example, as discussed above,processor 110 may be configured to perform process 1000 by executingsoftware or firmware stored in memory device 120, or may be configuredto perform process 1000 using dedicated hardware or one or more ASICs.

Processor 110 may be configured to receive first pixel information beingderived from a first external soft tissue image of the subject recordedat a first time (step 1010). Processor 110 may be configured, forexample, to perform step 1010 in the same manner as step 210, discussedabove. The first pixel information may be derived from a relationshipbetween pixels in a first group of pixels in the first external softtissue image. For example, the first group of pixels may correspond to aregion of interest in the first external soft tissue image. The regionof interest may be the entire first external soft tissue image or asubset thereof, depending on implementation-specific considerations. Forexample, in some embodiments, only a predetermined section of theacquired image may be of interest because the predetermined sectioncorresponds to a portion of the subject being screened for a symptom ofa genetic condition. The relationship between the pixels in the firstgroup of pixels may define, for instance, the spatial relationshipbetween the pixels in the first group of pixels, the relative values ofthe pixels in the first group, or any other informative relationship.

Processor 110 may be configured to analyze the first image information(step 1020), for example, by pre-analyzing a relationship (e.g.,relative values) between the pixels in the first group of pixels.Processor 110 may be configured, for example, to analyze a relationshipbetween the pixels in the first image information in the same manner asdescribed above with respect to steps 220-260. Optionally, however, onlyone analysis may be performed (e.g., only the analysis in steps 220-230)rather than the two (or more) analyses described in steps 220-250. Forexample, in some embodiments the analysis includes at least one ofanchored cells analysis, shifting patches analysis and relativemeasurements analysis. Moreover, as described above, in some embodimentsthe analysis includes a comparison of the first soft tissue imageinformation to an external soft tissue image of at least one individualhaving at least one of substantially the same age, ethnicity, and genderas the subject.

Processor 110 may be configured to receive second information reflectiveof a second external soft tissue image of the subject recorded at asecond time (step 1030). Processor 110 may be configured, for example,to perform step 1010 in the same manner as step 210 discussed above. Thesecond pixel information may be derived from a relationship betweenpixels in a second group of pixels in the second external soft tissueimage. For example, the second group of pixels may correspond to aregion of interest in the second external soft tissue image. The regionof interest may be the entire second external soft tissue image or asubset thereof, depending on implementation-specific considerations. Forexample, in some embodiments, only a predetermined section of theacquired image may be of interest because the predetermined sectioncorresponds to a portion of the subject being screened for a symptom ofa genetic condition. The relationship between the pixels in the secondgroup of pixels may define, for instance, the spatial relationshipbetween the pixels in the second group of pixels, the relative values ofthe pixels in the second group, or any other informative relationship.

The second time may be, for example, a predetermined amount of timeafter the first time or a non-scheduled time after the first time. Insome embodiments, processor 110 may be configured to send an alert thatsecond information reflective of a second external soft tissue image ofthe subject should be recorded. For example, if second informationreflective of a second external soft tissue image of the subject has notbeen received within a predetermined amount of time, an alert may besent to the subject's physician. As another example, if the analysis ofthe first information provided an indication that there is a lowprobability that a patient has a medical condition, an alert may be sentif second information reflective of a second external soft tissue imageof the subject has not been received within a predetermined amount oftime.

Processor 110 may be configured to analyze the second image information(step 1040), for example, by analyzing a relationship (e.g., relativevalues) between the pixels in the second group of pixels. Processor 110may be configured, for example, to analyze the second image informationin the same manner as described above with respect to steps 220-260.Optionally, however, only one analysis may be performed (e.g., theanalysis in steps 220-230) rather than the two (or more) analysesdescribed in steps 220-250. For example, in some embodiments theanalysis includes at least one of anchored cells analysis, shiftingpatches analysis and relative measurements analysis. Moreover, asdescribed above, in some embodiments the analysis includes a comparisonof the second soft tissue image information to an external soft tissueimage of at least one individual having at least one of substantiallythe same age, ethnicity, and gender as the subject.

In some embodiments, processor 110 is configured to apply the sametechnique to analyze the first external soft tissue image informationand to analyze the second external soft tissue image information. Forexample, the technique applied at the first time may be recorded andreused the next time a subject is imaged. Alternatively, in someembodiments, if the analysis of the first image information did notindicate a likelihood of any medical condition, the analysis may bechanged at the second time when the second external soft tissue imageinformation is analyzed. As another alternative, if, for example, theanalysis of the first image information indicated that there was alikelihood of a medical condition associated with a particular body part(e.g., an ear), the analysis of the second image information may befocused on the particular body part. In some embodiments, processor 110may be configured to alert a party associated with capturing the secondexternal soft tissue image to capture one or more images of theparticular body part. The second external soft tissue image thus mayinclude one or more images of the particular body part.

In certain embodiments, the first pixel information and/or the secondpixel information may be received from a location corresponding to astorage location associated with the subject. For example, the firstand/or second pixel information may be received from a cloud storagelocation associated with the subject, an electronic personal photo albumof the subject, and so forth. Further, pixel information may beperiodically received from one or more storage locations associated withthe subject. For example, the processor 110 may be configured to monitorthe pixel information associated with a subject over time byperiodically receiving updated pixel information corresponding toupdated external soft tissue images as the subject ages. This processmay be automated such that the updated pixel information isautomatically transferred to the processor from the one or more storagelocations, or manual such that the subject or another designated agentmanually initiates the transfers of the updated pixel information.

In some embodiments, the processor 110 may be configured to determine atime lapse between an original capture of the first external soft tissueimage and an original capture of the second external soft tissue image(step 1045). The original capture of each of the respective images maycorrespond to the time at which each image was first acquired by animage acquisition device (e.g., a camera). For example, the originalcapture of the first external soft tissue image may occur when a camerais used to capture an image of the subject when the subject is twelveyears old, and the original capture of the second external soft tissueimage may occur when a camera is used to capture an image of the subjectwhen the subject is fifteen years old. As such, the original capture ofthe image may correspond to the time when the image is initiallycaptured by an image acquisition device, which may or may not correspondto when the processor 110 receives or accesses the images.

In some embodiments, the processor 110 may be configured to determinethe time lapse based on metadata (e.g., geolocation data, timestamps,etc.) associated with one or both of the first external soft tissueimage and the second external soft tissue image. For example, when thefirst and/or second soft tissue image is acquired by an imageacquisition device (e.g., a camera), the image acquisition device mayembed metadata into a file corresponding to the first and/or second softtissue image. This embedded metadata may be accessed by the processor110 to determined, for example, an acquisition date, acquisition time,acquisition location, etc. Further, in some embodiments, the processor110 may be configured to account for one or more differences (e.g.,different time zones) in the metadata associated with each of the firstand second external soft tissue images.

Further, in certain embodiments, the processor 110 may determine thetime lapse between the original capture of the first and second externalsoft tissue images by implementing an age detection algorithm configuredto estimate the age of the subject in each of the images. For example,the age detection algorithm may be any algorithm, such as those known tothose skilled in the art, capable of identifying the subject's age basedon one or more factors in the image. The factors in the image mayinclude any factors indicative of aging, for example, number of depth ofwrinkles or lines, estimated skin firmness, or other changes in thesubject's skin or features.

Processor 110 may be configured to compare, based on the determined timelapse, the analysis of the first image information (e.g., first pixelinformation) with the analysis of the second image information (e.g.,second pixel information) (step 1050). For example, the first imageinformation and the second image information may include probabilityscores for a number of dysmorphic features. Processor 110 may beconfigured to determine changes in probability scores of the dysmorphicfeatures over time. The first image information and second imageinformation may also include, for example, probability scores for anumber of medical conditions. Processor 110 may be configured todetermine changes in probability scores of the medical conditions overtime.

As another example, the first image information and the second imageinformation may also include one or more severity scores of one or moredysmorphic features. For example, the first image information and thesecond image information may include one or more severity scores basedon one or more distances between feature points, angles formed byfeature points, ratios between distances, ratios between angles, and thelike. Thus, in some embodiments, processor 110 may be configured tomonitor progress of the medical condition to determine a change inseverity of an attribute over time.

Processor 110 may be configured to predict a likelihood that the subjectis affected by the medical condition based, at least in part, on thecomparison and/or the determined time lapse (step 1060). For example, inone embodiment, the processor 110 may be configured to predict, based onthe time lapse, a likelihood that the subject is affected by the medicalcondition based. For further example, probability scores and severitiesof one or more dysmorphic features at the first time, probability scoresand severities of one or more dysmorphic features at the second time,and/or changes in probability scores and severities of one or moredysmorphic features from the first time to the second time may be inputinto a classifier trained on, for example, positive and negativeexamples of a medical condition. If, for example, the severity orprobability scores associated with a set of dysmorphic featuresincreases from the first time to the second time, and the dysmorphicfeatures are associated with a medical condition, a relatively highlikelihood for the medical condition may be determined.

However, not all changes in severity will necessarily result in a highlikelihood for the medical condition. For example, as a child ages, somethe size of various dysmorphic features may be expected to change. Thus,in some embodiments, the change in severity of a dysmorphic feature maybe compared to known changes that occur from the subject's age at thefirst time to the subject's age at the second time (optionally, knownchanges that occur from the subject's age at the first time to thesubject's age at the second time may be examined in the context of atleast one of the subject's gender, ethnicity, and any other categorydescribing the subject). The known changes that occur from the subject'sage at the first time to the subject's age at the second time may bedetermined, for example, by analyzing images in a database anddetermining norms for a given patient population. Thus, in someembodiments, if the change in severity of a dysmorphic feature deviatesfrom an expected change, then a high likelihood that the subject isaffected by the medical condition may be predicted. If the change inseverity of a dysmorphic feature does not deviate from an expectedchange, then a low likelihood that the subject is affected by themedical condition may be predicted.

In some embodiments, processor 110 may increase a probability scoredetermined at the second time if an increase in probability score fromthe first time to the second time is determined. In some embodiments,processor 110 may decrease a probability score determined at the secondtime if a decrease in probability score from the first time to thesecond time is determined.

In some embodiments, a plurality of additional sets of information maybe received reflecting a plurality of additional external soft tissueimages recorded at a plurality of additional times. Processor 110 may beconfigured to analyze the plurality of additional images, compare theanalysis of the first soft tissue image information, the second softtissue image information, and the additional sets of soft tissue imageinformation, and predict the likelihood that the subject is affected bythe medical condition based on the comparison of the analysis of thefirst soft tissue image information, the second soft tissue imageinformation, and the additional sets of soft tissue image information

FIG. 11 illustrates an exemplary process 1100 that at least oneprocessor may be configured to perform. For example, as discussed above,processor 110 may be configured to perform process 1100 by executingsoftware or firmware stored in memory device 120, or may be configuredto perform process 1100 using dedicated hardware or one or more ASICs.

Processor 110 may be configured to receive first information reflectiveof an external soft tissue image of a first subject suspected of havingan unrecognized medical condition (step 1110). Processor 110 may beconfigured, for example, to perform step 1110 in the same manner as step210 discussed above.

Processor 110 may be configured to receive second information reflectiveof an external soft tissue image of a second subject suspected of havingan unrecognized medical condition (step 1120). Processor 110 may beconfigured, for example, to perform step 1120 in the same manner as step210 discussed above.

Processor 110 may be configured to compare the first image informationand the second image information (step 1130). For example, a relativemeasurements analysis may be used to generate a vector of relativemeasurements for the first image information and the second imageinformation. In some embodiments, the first image information isassociated with a new subject and the second image information isassociated with a previously presented individual. Thus, for example, asdepicted in FIG. 28, a vector of relative measurements associated withthe first image information may be compared against a set of vectors ofrelative measurements in a database (including the vector of relativemeasurements associated with the second image information). As anotherexample, processor 110 may be configured to receive first imageinformation from a first healthcare provider and receive the secondimage information from a second healthcare provider. Processor 110 mayenable the first healthcare provider to access image informationprovided by the second healthcare provider but deny access to text data(e.g., a patient name) provided by the second healthcare provider, andvice versa.

Processor 110 may be configured to determine, based on the comparison,that the first subject and the second subject are likely to possess thesame previously unrecognized medical condition (step 1140). For example,processor 110 may be configured to determine that the first subject andthe second subject are likely to possess the same previouslyunrecognized medical condition if the first image information and thesecond image information have a high degree of similarity to each otherand a high degree of dissimilarity to other images in the database. Thesimilarity may be determined, for example, based on a comparison of thevectors of relative measurements (e.g., as graphically depicted in thebottom right of FIG. 28, a distance may be determined from a vector ofrelative measurements associated with the first image information to avector of relative measurements associated with the second imageinformation). Processor 110 may enable the first healthcare provider tocommunicate with the second healthcare provider if it is determined thatthe first subject and the second subject are likely to possess the samepreviously unrecognized medical condition.

FIG. 12 illustrates an exemplary process 1200 that at least oneprocessor may be configured to perform. For example, as discussed above,processor 110 may be configured to perform process 1200 by executingsoftware or firmware stored in memory device 120, or may be configuredto perform process 1200 using dedicated hardware or one or more ASICs.

Processor 110 may be configured to use a computer-based external softtissue image analyzer to determine a likelihood that a specificindividual is affected by a medical condition for which at least onehealth service provider offers products or services relating to themedical condition, wherein a user of the computer-based external softtissue image analyzer is a healthcare professional (step 1210). Forexample, processor 110 may be configured to determine a likelihood thata specific individual is affected by a medical condition using the sameoperations as described above with respect to steps 210-260. Processor110 may be configured to access a database that includes products orservices offered by one or more health service providers and dataassociated the products or services with one or more related medicalconditions. Thus, after determining a likelihood of a medical condition,processor 110 may determine whether at least one health service provideroffers products or services relating to the medical condition.

Processor 110 may be configured to identify information about thehealthcare professional (step 1220). For example, processor 110 may beconfigured to identify information including one or more of healthcareprofessional contact information, education, expertise, training,experience with the medical condition, and the like.

Processor 110 may be configured to mediate communication between the atleast one health service provider and the healthcare professional basedon the likelihood that the specific individual is affected by themedical condition (step 1230). For example, mediating may includealerting the healthcare professional of the existence of at least one ofinformation regarding clinical trials, registries, diagnostics, andsecond opinions. An alert may be sent, for example, using a text messageto a telephone number of the healthcare professional, using an emailmessage to an email address of the healthcare professional, or using atelephone call to a telephone number of the healthcare professional. Thealert may provide the healthcare professional with an option to contact(e.g., using text message, email, or telephone communication) the healthservice provider. In some embodiments, processor 110 may be configuredto mediate if the likelihood is above a threshold. In some embodiments,processor 110 may be configured to mediate differently depending on thelikelihood. For example, if the likelihood of the medical condition ismoderate, then only the healthcare professional may receive acommunication. If the likelihood of the medical condition is high, thenboth the health service provider and the healthcare professional mayreceive a communication.

FIG. 13 illustrates an exemplary process 1300 that at least oneprocessor may be configured to perform. For example, as discussed above,processor 110 may be configured to perform process 1300 by executingsoftware or firmware stored in memory device 120, or may be configuredto perform process 1300 using dedicated hardware or one or more ASICs.

Processor 110 may be configured to maintain a database of external softtissue images of individuals having a medical condition (step 1310). Forexample, processor 110 may be configured to maintain a database inaccordance with any of the databases discussed above.

Processor 110 may be configured to receive from a healthcare provider animage feed of external soft tissue images of subjects from thehealthcare provider (step 1320). For example, processor 110 may beconfigured receive each image in the image feed in accordance with theoperations described above with respect to step 210. Moreover, processor110 may be configured to receive one or more dysmorphic featureannotations associated with each received image. The dysmorphic featureannotations may be generated by, for example the healthcare provider orby any of the other techniques described herein. Moreover, processor 110may be configured to receive one or more medical conditions associatedwith each received image. The medical condition may be generated by thehealthcare provider or by any of the other techniques described herein.

Processor 110 may be configured to use a computer-based external softtissue image analyzer to compare images in the feed with images in thedatabase (step 1330). For example, processor 110 may be configured tocompare each image in the feed with images in the database in accordancewith the operations described above with respect to steps 220-250.Optionally, however, only one analysis may be performed (e.g., theanalysis in steps 220-230) rather than the two (or more) analysesdescribed in steps 220-250. In some embodiments, the annotateddysmorphic features and medical conditions may be used to limit theanalysis. For example, in some embodiments, only a likelihood of theannotated dysmorphic features may be determined. Likewise, in someembodiments, only a likelihood of the medical conditions received witheach image may be determined.

Processor 110 may be configured to, based on the comparison, determinewhen an image of a subject in the feed meets a threshold of being likelyto be affected by a medical condition (step 1340). For example,processor 110 may be configured to determine when an image of a subjectin the feed meets a threshold of being likely to be affected by amedical condition in accordance with the operations described above withrespect to step 260.

Processor 110 may be configured to alert the healthcare provider whenthe image of the subject meets the threshold (step 1350). For example,processor 110 may be configured to send an alert to the healthcareprovider using, for example, a text message, a telephone call, an email,and the like. As another example, processor 110 may be configured topresent an alert on a display of the device that is used by thehealthcare provider to capture the image. The alert may include, forexample, a patient name or other patient identifier and data regardingthe medical condition that triggered the alert. The data regarding themedical condition may include, for example, the name of the medicalcondition, dysmorphic features associated with the medical condition,suggested treatments for the medical condition, suggested additionaltests for the medical condition, etc.

Processor 110 may be configured to add the image to the database ofexternal soft tissue images based on whether the subject actuallypossesses the medical condition (step 1360). For example, processor 110may be configured to receive confirmation from the healthcare providerthat the subject possesses the medical condition. The confirmation maybe based on, for example, the additional tests included in the alert.

In some embodiments, the images of the confirmed subjects and/or dataderived from the confirmed subjects may be linked with the medicalcondition. For example, the database may be configured to annotate theconfirmed subject data with the medical condition. The confirmed subjectdata may then be used to train one or more classifiers for the medicalcondition as a positive example of the medical condition. Likewise,processor 110 may also be configured to receive an indication from thehealthcare provider that the subject does not possess the medicalcondition. The negative subject data may then be used to train one ormore classifiers for the medical condition as a negative example (i.e.,a control) or a false positive example.

FIG. 14 illustrates an exemplary process 1400 that at least oneprocessor may be configured to perform. For example, as discussed above,processor 110 may be configured to perform process 1400 by executingsoftware or firmware stored in memory device 120, or may be configuredto perform process 1400 using dedicated hardware or one or more ASICs.

In some embodiments, prior to implementing step 1410, the processor 110may be configured to receive populational electronic informationreflective of populational sets of values corresponding to pixels in aplurality of cranio-facial external soft tissue images associated with aplurality of geographically dispersed individuals having a medicalcondition, such as a genetic disorder. Each populational set of valuesmay correspond to relationships between at least one group of pixels ineach of the cranio-facial external soft tissue images. For example, eachcranio-facial soft tissue image may be a digital image having an arrayof pixels, with each pixel having a populational value. The populationalsets of values may be pixels values that are reflective of relationshipsbetween one or more groups of pixels in each of the cranio-facialexternal soft tissue images. For instance, in some applications, the atleast one group of pixels may correspond to a region of interest in thecranio-facial external soft tissue images (e.g., one that corresponds toan anatomical location predicted to have an abnormality), and thepopulational sets of values may correspond to relationships between thepixels in the selected region of interest. As such, the populationalsets of values may be actual pixel values corresponding to the pixels inthe digital images, or some derivative thereof, which may, for example,be normalized across pixels in the at least one group of pixels (e.g.,by setting the lowest pixel value to zero and adjusting the other pixelvalues accordingly).

Once the populational electronic information is received, the processor110 may then use the populational sets of values to generate electroniccharacterizations of each of the plurality of cranio-facial externalsoft tissue images. The electronic characterizations may be any type ofelectronic representation of a subset or full set of the plurality ofcranio-facial external soft tissue images. For example, in someembodiments, the electronic characterizations may be pixel valuescorresponding to a selected subset of the plurality of cranio-facialexternal soft tissue images and/or one or more selected portions of eachof the plurality of cranio-facial external soft tissue images.

Processor 110 may be configured to associate, in an electronic database,the electronic characterizations of the plurality of external softtissue images of individuals (e.g., geographically dispersedindividuals) having a medical condition, such as one or more geneticdisorders (step 1410). For example, processor 110 may be configured toassociate a plurality of external soft tissue images of individualshaving a medical condition in an electronic database in accordance withthe operations described above.

Processor 110 may be configured to analyze the electroniccharacterizations of the plurality of external soft tissue images,and/or the plurality of external soft tissue images, which may becranio-facial images, to identify at least one populational predictorlocation associated with an external soft tissue attribute (e.g., adysmorphic feature) predictive of the medical condition, such as the oneor more genetic disorders (step 1420). For example, processor 110 may beconfigured to detect one or more dysmorphic features using theoperations described above. Processor 110 may be configured to determinewhether a number of the external soft tissue images that are associatedwith individuals having a medical condition are also associated with oneor more of the same dysmorphic features at, for example, a same orsimilar location.

Processor 110 may be configured to receive subject-related electronicinformation for a subject undiagnosed with the medical condition (step1430). For example, processor 110 may be configured to receive anexternal soft tissue image of a subject in accordance with the sameoperations described with respect to step 210. In other embodiments, thesubject-related electronic information may be reflective ofsubject-related sets of values corresponding to pixels of acranio-facial external soft tissue image of the subject. Thesubject-related sets of values may correspond to relationships betweenat least one group of pixels in a cranio-facial external soft tissueimage of the subject. For example, the cranio-facial external softtissue image of the subject may be a digital image having an array ofpixels, with each pixel having a subject-related value. Thesubject-related sets of values may be pixel values that are reflectiveof relationships between one or more groups of pixels in thecranio-facial external soft tissue image of the subject. For instance,in some applications, the at least one group of pixels may correspond toa region of interest in the cranio-facial external soft tissue image ofthe subject (e.g., one that corresponds to an anatomical locationpredicted to have an abnormality), and the subject-related sets ofvalues may correspond to relationships between the pixels in theselected region of interest. As such, the subject-related sets of valuesmay be actual pixel values corresponding to the pixels in the digitalimage, or some derivative thereof, which may, for example, be normalizedacross pixels in the at least one group of pixels (e.g., by setting thelowest pixel value to zero and adjusting the other pixel valuesaccordingly).

Processor 110 may be configured to analyze the subject-relatedelectronic information, and/or the subject's external soft tissue imageto identify the subject predictor location corresponding to thepopulational predictor location (step 1440). For example, processor 110may be configured to identify a region in the subject's external softtissue image information corresponding to or containing the dysmorphicfeature identified in step 1420.

Processor 110 may be configured to compare information associated withpixels corresponding to the populational predictor location of theplurality of external soft tissue images with information associatedwith pixels corresponding to the subject predictor location in thesubject's external soft tissue image (step 1450) and to determinewhether a common dysmorphology exists at the populational predictorlocation of at least some of the plurality of external soft tissueimages and the subject predictor location of the subject's external softtissue image (step 1460). For example, processor 110 may be configuredto determine whether the region of the subject's external soft tissueimage containing the predictor location is similar to one or more of theregions containing the predictor location in the plurality of externalsoft tissue images. For example, processor 110 may be configured todetermine that one or more of the identified dysmorphic features arecontained in the region of the subject's external soft tissue imagecontaining the predictor location.

Processor 110 may be configured to predict, based on the determination,whether the subject has the medical condition, such as a geneticdisorder (step 1470). For example, if one or more of the identifieddysmorphic features are detected at the predicted location in thesubject's external soft tissue image, then processor 110 may predictthat the subject has the medical condition. Similarly, processor 110 maybe configured to predict that the subject has the medical condition if asufficient number of dysmorphic features are detected in a sufficientnumber of predictor locations.

FIG. 15 illustrates an exemplary process 1500 that at least oneprocessor may be configured to perform. For example, as discussed above,processor 110 may be configured to perform process 1500 by executingsoftware or firmware stored in memory device 120, or may be configuredto perform process 1500 using dedicated hardware or one or more ASICs.

Processor 110 may be configured to receive information reflective of anexternal soft tissue image of the subject (step 1510). Processor 110 maybe configured, for example, to perform step 1510 in the same manner asstep 210, discussed above.

Processor 110 may be configured to analyze the external soft tissueimage information for a dysmorphology (step 1520). For example,processor 110 may be configured to analyze the external soft tissueimage information using the same or substantially the same operationsdescribed above with respect to steps 220-250. Optionally, however, onlyone analysis may be performed (e.g., the analysis in steps 220-230)rather than the two (or more) analyses described in steps 220-250.

Processor 110 may be configured to determine a plurality of potentialmedical conditions associated with the dysmorphology (step 1530). Forexample, processor 110 may be configured to determine a plurality ofpotential medical conditions using the same operations described abovewith respect to step 260.

Processor 110 may be configured to generate a list of tests to beperformed in order to determine whether the individual possesses atleast one of the plurality of potential medical conditions (step 1540).For example, processor 110 may be configured to determine all testshaving diagnostic value for a potential medical condition. In someembodiments, processor 110 may be configured to generate a list of testsbased on at least one of the price of the test, the accuracy of thetest, and the compatibility of the test with the plurality of potentialmedical condition.

In some embodiments, processor 110 may receive information reflective ofprevious tests that were selected in response to a generated list. Basedon the received information, processor 110 may be configured to favorthe selected tests. For example, if one test is more expensive thananother test, processor 110 may be initially configured to output thecheaper test first. However, if processor 110 receives informationindicating that the more expensive test is more widely used, then themore expensive test may be included first for subsequent generatedlists.

FIG. 16 illustrates an exemplary process 1600 that at least oneprocessor may be configured to perform. For example, as discussed above,processor 110 may be configured to perform process 1600 by executingsoftware or firmware stored in memory device 120, or may be configuredto perform process 1600 using dedicated hardware or one or more ASICs.

Processor 110 may be configured to receive information reflective of anexternal soft tissue image of the subject (step 1610). Processor 110 maybe configured, for example, to perform step 1610 in the same manner asstep 210, discussed above.

Processor 110 may be configured to analyze the external soft tissueimage information (step 1620). For example, processor 110 may beconfigured to analyze the external soft tissue image information usingthe same or substantially the same operations described above withrespect to steps 220-250. Optionally, however, only one analysis may beperformed (e.g., the analysis in steps 220-230) rather than the two (ormore) analyses described in steps 220-250.

Processor 110 may be configured to determine, based on the analysis, aprobability that a dysmorphic feature exists (step 1630). For example,processor 110 may be configured to determine a probability score of adysmorphic feature using operations described above.

Processor 110 may be configured to assign a severity score to thedysmorphic feature (step 1640). For example, processor 110 may beconfigured to assign a severity score to a dysmorphic feature based on arelative measurements analysis. For example, the severity of a longphiltrum dysmorphic feature may be measured by the ratio of the lengthdefined by feature points associated with the philtrum to lengthsdefined by feature points associated with one or more of the nose,mouth, and height of a face of the subject. In some embodiments, theseverity score is determined after a probability score is determined.For example, in some embodiments, if the probability score for adysmorphic feature is above a threshold, the severity score may then bedetermined. The severity score may be determined as a function of aprobability score for the dysmorphic feature (e.g., a higher probabilityscore may receive a higher severity score) or by using a secondclassifier trained, for example, on data from individuals known to havevarious predetermined severities of the dysmorphic feature.

Processor 110 may be configured to predict whether the dysmorphicfeature is indicative of a medical condition based on the severity score(step 1650). For example, a classifier may be trained on positive andnegative examples of the medical condition to receive probability scoresand severity scores associated with a set of dysmorphic features and tooutput a probability score for the medical condition. Processor 110 may,for example, use the trained classifier to determine whether theseverity score of the dysmorphic feature is indicative of a medicalcondition.

FIG. 17 illustrates an exemplary process 1700 that at least oneprocessor may be configured to perform. For example, as discussed above,processor 110 may be configured to perform process 1700 by executingsoftware or firmware stored in memory device 120, or may be configuredto perform process 1700 using dedicated hardware or one or more ASICs.

Processor 110 may be configured to receive information reflective of anexternal soft tissue image of the subject (step 1710). Processor 110 maybe configured, for example, to perform step 1710 in the same manner asstep 210, discussed above.

Processor 110 may be configured to analyze the external soft tissueimage information and identify a first dysmorphic attribute and a seconddysmorphic attribute (step 1720). For example, processor 110 may beconfigured to analyze the external soft tissue image information andidentify at least two dysmorphic features in the external soft tissueimage information using the same or substantially the same operationsdescribed above with respect to steps 220-250. Optionally, however, onlyone analysis may be performed (e.g., the analysis in steps 220-230)rather than the two (or more) analyses described in steps 220-250.

Processor 110 may be configured to determine that the first dysmorphicattribute is less likely to be a predictor of the medical condition thanthe second dysmorphic attribute (step 1730). For example, processor 110may be configured to determine that the first dysmorphic attribute isless likely to be a predictor of the medical condition than the seconddysmorphic attribute based on information that the first dysmorphicattribute typically does not coincide with the second dysmorphicattribute. As another example, processor 110 may be configured todetermine that the first dysmorphic attribute is less likely to be apredictor of the medical condition than the second dysmorphic attributebased on information that the first dysmorphic attribute is commonamongst family members of the subject. As another example, processor 110may be configured to determine that the first dysmorphic attribute isless likely to be a predictor of the medical condition than the seconddysmorphic attribute based on information that the first dysmorphicattribute is common amongst members of the individual's ethnicity. Asanother example, processor 110 may be configured to determine that thefirst dysmorphic attribute is less likely to be a predictor of themedical condition than the second dysmorphic attribute based oninformation that the first dysmorphic attribute is common amongstmembers of the individual's gender.

Processor 110 may be configured to predict whether the subject is likelyto be affected by a medical condition, wherein, during the prediction,the first dysmorphic attribute is discounted (step 1740). For example,processor 110 may be configured to predict whether the subject is likelyto be affected by a medical condition using substantially the sameoperations described above with respect to step 260. However, theclassifier used to make the prediction may, for example, be configuredto ignore the first dysmorphic attribute or provide it with a reducedweight.

FIG. 29 illustrates an exemplary process 2900 that at least oneprocessor may be configured to perform. For example, as discussed above,processor 110 may be configured to perform process 2900 by executingsoftware or firmware stored in memory device 120, or may be configuredto perform process 2900 using dedicated hardware or one or more ASICs.

Processor 110 may be configured to process a plurality of vectorsderived from one or more images of geographically dispersed individualshaving a medical condition (step 2910). For example, in one embodiment,the processor 110 may be configured to process a plurality of vectorsderived from information extracted from cranio-facial images ofgeographically dispersed individuals having a genetic condition. Morespecifically, in some embodiments, the processor 110 may format andstore the plurality of vectors in an electronic database for furtheruse. In some embodiments, in the electronic database, the processor 110may group vectors associated with like medical conditions (e.g., geneticdisorders) together for future comparisons.

The vectors may be any type of image descriptors, for example, asdescribed above with reference to the anchored cells analysis, shiftingpatches analysis, and relative measurements analysis. In additional tothese image descriptors, extracted information from the image mayinclude metadata, for example, corresponding to the age, gender, orother characteristics of the subject or image.

The processor 110 may be further configured to receive or access one ormore vectors extracted from a cranio-facial image of a subject and tocompare the one or more vectors to the plurality of vectors (step 2920).In some embodiments, the one or more vectors may be received from atleast one healthcare provider of the subject. Healthcare providers mayinclude, but are not limited to, one or more of physicians, geneticists,genetic counselors, medical researchers, dentists, pharmacists,physician assistants, nurses, advanced practice registered nurses,surgeons, surgeon's assistants, athletic trainers, surgicaltechnologists, midwives, dietitians, therapists, psychologists,chiropractors, clinical officers, social workers, phlebotomists,occupational therapists, physical therapists, radiographers, respiratorytherapists, audiologists, speech pathologists, optometrists, emergencymedical technicians, paramedics, medical laboratory scientists, medicalprosthetic technicians, and a wide variety of other human resourcestrained to provide some type of health care service, such as managers ofhealth care services, clinical directors, lab directors, healthinformation technicians, and other assistive personnel and supportworkers. Healthcare providers may include individuals working inhospitals, health care centers, laboratories, and other service deliverypoints, as well as in academic education, research groups, andadministration and pharmaceutical companies.

In some embodiments, the processor 110 may implement a cranio-facialimage or vector comparison of the one or more vectors with the pluralityof vectors without reference to (i.e., without accessing, using, orvisually comparing) the discernible cranio-facial image (e.g., thecomplete set of data encoding the acquired cranio-facial image) of thesubject with the cranio-facial images of the geographically dispersedindividuals. The foregoing feature may offer one or more advantages oversystems that perform a cranio-facial image comparison between the imagesof the plurality of geographically dispersed individuals and thesubject, respectively. For example, by making a comparison on thevector-level instead of the image-level, the privacy of the subjectand/or the plurality of geographically dispersed individuals may bepreserved.

The processor 110 may be further configured to determine when a vectorin the one or more vectors is indicative of the subject meeting athreshold of being likely to be affected by a medical condition (step2930). For example, the subject may be undiagnosed with a geneticcondition, and the plurality of geographically dispersed individuals maybe diagnosed with the genetic condition. As such, the threshold valuemay be a level of similarity between the vector in the one or morevectors and a vector in the plurality of vectors. In some embodiments,if the threshold level is met, the processor 110 may be furtherconfigured to alert a healthcare provider, for example, by displayingthe alert on an output device. In still further embodiments, if thethreshold level (or another predictive threshold level) is met theprocessor 110 may be configured to add the vector of the one or morevectors to the electronic database. For example, if the vector of theone or more vectors is similar enough to the plurality of vectors knownto be associated with the genetic condition, there may be enoughsimilarity to use the vector of the one or more vectors in futurecomparisons.

In some embodiments, the processor 110 may receive information from ahealthcare provider regarding whether the subject has the geneticcondition. If the subject does have the genetic condition, the processor110 may add the vector of the one or more vectors to the electronicdatabase. In other embodiments, the processor 110 may add the vector ofthe one or more vectors to the electronic database only if the subjectis determined to have the genetic disorder, regardless of comparison tothe threshold in step 2930.

Certain features which, for clarity, are described in this specificationin the context of separate embodiments, may also be provided incombination in a single embodiment. Conversely, various features which,for brevity, are described in the context of a single embodiment, mayalso be provided in multiple embodiments separately or in any suitablesub-combination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Particular embodiments have been described. Other embodiments are withinthe scope of the following claims.

1-19. (canceled)
 20. A computer-implemented method for determining froman image of external cranio-facial soft tissue whether a subject islikely to have a dysmorphic feature, the computer-implemented methodcomprising: receiving, with processing circuitry, the cranio-facialexternal soft tissue image of the subject; accessing cranio-facialexternal soft tissue images of a plurality of individuals; processing,with the processing circuitry, the external soft tissue image of thesubject, with respect to the external soft tissue images of theindividuals, wherein the processing includes one or more of an anchoredcells analysis, shifting patches analysis, and relative measurementsanalysis; determining, with the processing circuitry, based on at leastone of the anchored cells analysis, the shifting patches analysis, andthe relative measurements analysis, at least one dysmorphic featureincluded in the cranio-facial external soft tissue image of the subject;accessing a database of descriptors of dysmorphic features; andselecting, based on the determining by at least one of the anchoredcells analysis, the shifting patches analysis, and the relativemeasurements analysis, at least one of the descriptors associated withthe at least one determined dysmorphic feature.
 21. Thecomputer-implemented method of claim 20, further including displaying acranio-facial image of the subject together with the at least onedescriptor of the determined at least one dysmorphic feature and anindication in the image of a location of the at least one dysmorphicfeature.
 22. The computer-implemented method of claim 20, whereinprocessing the cranio-facial external soft tissue image of the subjectincludes identifying at least one of potentially dysmorphic features,and assigning, with the processing circuitry, a probability score to atleast one of identified potential dysmorphic features.
 23. Thecomputer-implemented method of claim 20, wherein accessing the databaseof descriptors includes accessing a reference list of descriptors storedin at least one memory device, the reference list of descriptorsincluding at least one medical ontology-based descriptors.
 24. Thecomputer-implemented method of claim 23, further comprising mappingadditional descriptors to the medical ontology-based descriptors, andadding the additional descriptors to the reference list.
 25. Thecomputer-implemented method of claim 23, further comprising updating,with the processing circuitry, the reference list stored on the at leastone memory device after the at least one medical ontology is updated.26. The computer-implemented method of claim 20, wherein the at leastone of the descriptors associated with the at least one determineddysmorphic feature includes a textual description and a list of synonymsof the textual description.
 27. The computer-implemented method of claim20, wherein being affected by a genetic disorder includes at least oneof possessing a genetic syndrome and being a carrier of the geneticsyndrome.
 28. The computer-implemented method of claim 20, whereinprocessing the cranio-facial external soft tissue image of the subjectincludes evaluating the cranio-facial external soft tissue image of thesubject based on a plurality of objective criteria, including at leastone of age, gender, and ethnicity.
 29. The computer-implemented methodof claim 20, wherein the descriptors comprise a list of words includingterms that are compatible with a plurality of databases for searchingmedical conditions.
 30. The computer-implemented method of claim 20,wherein the descriptors comprise a list of words including at least onedescription of a general appearance associated with a medical condition.31. The computer-implemented method of claim 20, wherein the externalcranio-facial soft tissue image includes at least one of a frontal view,a lateral view, an angled view, a top view, and a back view.
 32. Asystem for determining from an image of external cranio-facial softtissue whether a subject is likely to be affected by a genetic disorder,the system comprising: at least one memory for storingcomputer-executable instructions; and at least one processor configuredto execute the stored instructions to: receive the cranio-facialexternal soft tissue image of the subject; access cranio-facial externalsoft tissue images of a plurality of individuals; process thecranio-facial external soft tissue image of the subject, with respect tothe cranio-facial external soft tissue images of the plurality ofindividuals, wherein the processing includes one or more of an anchoredcells analysis, shifting patches analysis, and relative measurementsanalysis; determine, based on at least one of the anchored cellsanalysis, the shifting patches analysis, and the relative measurementsanalysis, at least one dysmorphic feature included in the cranio-facialexternal soft tissue image of the subject; access a database ofdescriptors of dysmorphic features; and select, based on the determiningby at least one of the anchored cells analysis, the shifting patchesanalysis, and the relative measurements analysis, at least one of thedescriptors associated with the at least one determined dysmorphicfeature.
 33. The system of claim 32, wherein the at least one processoris further configured to maintain a reference list of descriptors in theat least one memory sourced from one or more of a plurality of databasesfor searching medical conditions, and to access the descriptors from thereference list.
 34. The system of claim 33, wherein the at least oneprocessor is further configured to reconfigure the reference list ofdescriptors in the at least one memory after each of the plurality ofdatabases is updated.
 35. The system of claim 32, wherein the at leastone processor is further configured to output the selected at least oneof the descriptors and an indication on a cranio-facial image of thesubject of a location of the dysmorphic feature associated with thedescriptor.
 36. The system of claim 32, wherein the accessed database ofdescriptors includes a list of words associated with dysmorphicfeatures.
 37. The system of claim 32, wherein the at least one processoris further configured to assign a probability score to each dysmorphicfeature.
 38. A non-transitory computer-readable medium for determiningfrom an image of external cranio-facial soft tissue whether a subject islikely to be affected by a genetic disorder, which comprisesinstructions that, when executed by at least one processor, cause the atleast one processor to perform operations including: receiving, withprocessing circuitry, the cranio-facial external soft tissue image ofthe subject; accessing cranio-facial external soft tissue images of aplurality of individuals; processing, with the processing circuitry, thecranio-facial external soft tissue image of the subject, with respect tothe cranio-facial external soft tissue images of the plurality ofindividuals, wherein the processing includes one or more of an anchoredcells analysis, shifting patches analysis, and relative measurementsanalysis; determining, with the processing circuitry, based on at leastone of the anchored cells analysis, the shifting patches analysis, andthe relative measurements analysis, at least one dysmorphic featureincluded in the cranio-facial external soft tissue image of the subject;accessing a database of descriptors of dysmorphic features; andselecting, based on the determining by at least one of the anchoredcells analysis, the shifting patches analysis, and the relativemeasurements analysis, at least one of the descriptors associated withthe at least one determined dysmorphic feature.